How to ban IP address behind NAT


I am creating WebSocket server with rust and tokio and I want to prevent DDos attacks and spams.

So I thought of creating HashMap and inserting IP address which i suspect are trying to do spamming or DDos Attack but will this also ban other innocent users sharing same NAT network with attacker ?

If I ban IP address and port combination, will the attacker just use other port?