How does Tor preserve anonymity if it uses normal Internet Routing?

I’m studying Tor and Onion Routing and I don’t understand how it preserves anonymity if the Internet routing is still done using public ip addresses.

Let’s suppose we have the following Tor circuit: Tor Browser -> A -> B -> C -> Server. If someone follows the traffic from relay to relay then the anonymity is broken. Even though it uses 3 layers of encryption the routing is done by public ip addresses which are in clear text in the ip header.

Or when the server responds back it sends the packets to the public ip address of C. Some authority could follow to route from the server to C to B to A to the client and knows that the client is communicating with the server.

Can anyone say if I’m right? Or the entire security of Tor is based on the fact that no one can ever control all 3 relays (or statistically is very improbable)?