How can we limit access to a single computer?

We would like to limit access to a web server (and eventually other services on the computer) to individuals that have been authorized access. Of course we don’t trust passwords so we think certificates are the right answer.

There are hundreds of these servers. Access to any one server should NOT provide access to any other server. The access should be to only the single server. (Access will also be time limited for additional security).

How can we implement these security requirements?

We are currently on a path that would involve creating individual CAs for each server. The server would require mutual authentication for the server and client. The server and client certs would be signed by the unique CA for each server.

Is there an alternative? Perhaps one that does not involve creating many CAs?

Thanks for you advice.

FYI — The servers are all running Linux.