If you want to learn how to fight you can get by with mediocre boxing/stand up and a solid ground game.
If you want to just box, there's nothing wrong with that, but boxing is just that...boxing. It isn't fighting.
Any dumbass can land a lucky punch even against the best boxers. If you stand up with a guy in the street who isn't trained, even if you are a pretty good boxer, he can still land that lucky punch and fuck you up.
However, if you are great on the ground there is very little an untrained asshole can do to stop you from getting him on the ground. Once you are there it's like choking out or throwing an armbar on a child.
It's that easy.
My recommendation if you don't want to devote your life to fighting but still be able to handle yourself dependably?
Join a good MMA gym. Spend some of your time learning boxing and Muy Thai. You need it. This goes triple of you end up fighting someone who is actually trained.
Devote most of your time to wrestling and BJJ. You will learn the basics of Judo in any good wrestling program.
When you first start you will be everyone's bitch. Even guys smaller than you. Even guys younger than you. Anyone with six months of real training, you are their bitch and there is nothing you can do about it.
After a year however you won't have to worry about anyone in the street unless you run into someone else who has trained.
Interestingly enough, you won't feel like a badass. You will have been forced to tap and hit in the head so much by better boxer/kickboxers that you will understand your limits and how fragile you are.
You will assume other guys on the street are trained and might be better than you.
You have been taken to the point of death and only saved by tapping so many times by now the idea of fighting unless you truly have to is pretty fucking scary.
I've got eight years of training at a gym in Houston. Take my advice or don't.