As part of moving from few on-premise monoliths to multiple on-premise microservices, I’m trying to improve the situation where database passwords and other credentials are stored in configuration files in /etc.
Regardless of the the technology used, the consumer of the secrets needs to authenticate with a secret store somehow. How is this initial secret-consumer-authenication trust established?
It seems we have a chicken-and-egg problem. In order to get credentials from a server, we need to have a
/etc/secretCredentials.yaml file with a cert, token or password. Then I (almost) might as well stick to the configuration files as today.
If I wanted to use something like HashiCorp Vault (which seems to be the market leader) for this, there is a Secure Introduction of Vault Clients article. It outlines three methods:
- Platform Integration: Great if you’re on AliCloud, AWS, Azure, GCP. We’re not
- Trusted Orchestrator: Great if you’re using Terraform, Puppet, Chef. We’re not
- Vault Agent: The remaining candidate
When looking at the various Vault Auth Methods available to the Vault Agent, they all look like they boil down to having a cert, token or password stored locally. Vault’s AppRole Pull Authentication article describes the challenge perfectly, but then doesn’t describe how the app gets the
The only thing I can think of is IP address. But our servers are all running in the same virtualization environment, and so today, they all have random IP addresses from the same DHCP pool, making it hard to create ACLs based on IP address. We could change that. But even then, is request IP address/subnet sufficiently safe to use as a valid credential?
We can’t be the first in the universe to hit this. Are there alternatives to having a
/etc/secretCredentials.yaml file or ACLs based on IP address, or is that the best we can do? What is the relevant terminology and what are the best-practices, so we don’t invent our own (insecure) solution?