How does it affect the game if not everything spoke Common?

Going through the bestiary DBs, you find that most monsters that speak a language, almost all speak Common as well. (Or have some form of telepathy or truespeech.)

From the data my friend gathered, he determined there are about 75% of speaking creatures speak Common. Of those that don’t, 20% are elementals or variants of them, so this leaves only about 20% of creatures that can speak that don’t speak common.

What would the consequences be of removing Common from a large majority of creatures? My goal is to give players more immersion (not everything understands them if its more abnormal or abstract from common folk) and letting those that pick extra languages feel like they are more useful tools, without overshadowing someone who has a lower Int character, but may still want to be a social butterfly.