One of my players want to carry his familiar on his body and says it should not be targeted by enemies because it can hide on his body?

One of my players (of a rogue) said that he heard of DM’s allowing the familiar to be carried on the players body. He use his familiar to help him in combat and get advantage on the attack roll.

I said it’s okay for me and allowed it. But later in fight I played a goblin and obviously attacked his familiar in his pocket, to take his advantage away. He said that it shouldn’t be possibly because his familiar hides in a pocket and can’t be targeted. I didn’t allow that because it would obviously break the game. He said that many DM’s bend the rule like this and allow this.

I would like to hear whether anyone has ever allowed this change of the rules, and whether it did or will break the game.