There are two sides to the empty hand training coin: Self Mastery and Breaking People. The two are not necessarily related, but they can be *if you are diligent in your training and honest with yourself*.
Self Mastery is fun, and it keeps you going for the long haul. It allows you to explore facets of your particular art when you start getting bored with training. The truth is, you can do this with yoga or basket weaving or whatever, BUT if you do it with a martial art, you will occasionally bump into item two, "breaking people". Most martial arts contain either useless fluff, or are hyper focused on one idea at the exclusion of all else. For example, one of my arts contained a significant number of techniques that revolved around defending yourself against a spear when you only had a sword. Sure, you might find a modern application, but lets face it, that's a low probability. Conversely, BJJ is hyper focused on winning by submission. Submission is only useful in some contexts, and can be detrimental in others.
About breaking people. Breaking people is physically easy. It really doesn't take much skill at all, and if you have weapons, even less than that. Getting your attacker into a situation where he can't stop you from breaking him, well, that's a different thing entirely. You can learn almost everything you need to know about breaking people in a very short amount of time, which also makes this sort of training a very bad business model. Most places that charge you to train won't teach you much about breaking people, even though they say that is what they are doing.
Lets tie this all together. Adding more arts to your repertoire is very valuable because almost all of the martial arts have several hidden gems. For example, boxers are great hitters, and hitting hard is a good social skill. BJJ practitioners know that you can be out of strength and out of air against someone bigger than you, and you still have a good chance of winning. Wrestlers know about pressure and keeping their attacker on the defensive. So if you want exercise, comradery AND to know how to break people, look at what you are studying now and figure out what is missing. Spend some time filling in those holes in your knowledge. Just don't lose sight of your overall objective, whatever that may be for you.
(For an excellent read on the subject, check out "Meditations on Violence" by Rory Miller.
