Can someone immediately pick up their weapon after being disarmed by the Battle Master fighter’s Disarming Attack maneuver?

The Battle Master fighter’s Disarming Attack maneuver can make someone drop a weapon at their feet. The only advantage is if others want to dash away or you need to remove a reaction parry from the opponent before they get their own turn.

When the bad guy’s turn comes around, they pick it up (free interaction with an object) and go about their business.

Do I have that right?

To my understanding, you wouldn’t be able to use your own free object interaction to pick up and/or throw away their weapon (unless you have movement left over, and you’re either two sizes smaller than your opponent or you’re a halfling) because the weapon is in the enemy’s space – not yours.