What do I do if my group isn’t gelling?

It’s my first time playing D&D with a bunch of other first-timers (using the starter kit). One of the players is steamrolling over other players and his character is insufferable. What do I do if my group isn’t gelling?

After our last session, one of the players apparently told our DM, "It seems like we’re not all looking for the same kind of game," or something like that.

Three of us seem to be more interested in thinking around problems and the other two want to either smash open heads while yelling, "For justice!" or are making things weirdly sexual.

It’s kind of funny when the count says, "I am excellent at persuading people. I persuade him!" How do you persuade him? "With the dice!" But how? "With my mustache!" Sure. Sure! It is kind of funny. But it is also a very 2 dimensional character who takes up a lot of space and is getting on my nerves.

I might be a little more upset because this person is a family member who I’m rebuilding my relationship with, and I thought this could be a way for us to hang out in a light, fun way. He’s made a character, though, that is exaggerating the punitive justice, black-and-white thinking, my-way-or-the-highway attitude that makes it so hard to be around him in the first place. One of my friends said, "It seems like he thinks of himself as the protagonist". It just doesn’t make for a great group experience.

I feel pretty dumb for thinking this could work.

His wife is the player who’s making things very sexual, so that’s another element of "maybe funny, but mostly strange" that is in the mix.

Is there a way to turn this situation around? For those who have played in a game with tension like this: How do you break it down or adjust it? Or do you just give it up as a failed experiment?