
Ask HN: How is your company storing secrets in EC2? - misiti3780
What services&#x2F;encryption&#x2F;etc is your company using to store things like API keys and other secrets (strings that cant be hashed)? I see services like these:<p>https:&#x2F;&#x2F;aws.amazon.com&#x2F;kms&#x2F;<p>https:&#x2F;&#x2F;aws.amazon.com&#x2F;secrets-manager&#x2F;<p>https:&#x2F;&#x2F;medium.com&#x2F;@nqbao&#x2F;how-to-use-aws-ssm-parameter-store-easily-in-python-94fda04fea84<p>Just wondering what the best practices is in 2019
======
chrismeller
You’re confusing services and their intentions.

KMS provides encryption. Yes it can store small values for you, but that’s it
- encrypt this, decrypt that, rotate this key. Everything else is your job.
This is the low level service.

Secrets manager builds upon this. More access controls, some built in logic
for rotation of secrets, etc.

Which is correct for you depends upon what kind of secret you’re storing, what
you do with it, how often and how you need to rotate or regenerate it...
dozens of factors.

Basically, KMS is the key master. It manages the keys to the data and will
begrudgingly hang on to small bits of paper you’ve asked it to encrypt, but it
does nothing else. Data in, encryption, data out, decryption.

If an API key is one of those bits of data that’s fine. It knows not and cares
not. Need to rotate it? Pull it out, decrypt it, rotate it, encrypt it, and
shove it back in - all that is your business.

Secrets Manager is the gate keeper. It needs the key master but it knows more
about who and what is passing through and will help you out from time to time.
Stored your most recent allegiance to Zuul but you haven’t reaffirmed it
recently? He’ll be a bro and go ahead and send it back in for you and store
the new allegiance until you need it again.

------
laurentl
We’ve been using SSM parameter store to store our secrets (as secure strings).
To manage access in a more granular manner we use a namespace approach where
the parameters are prefixed with the service or API they’re attached to, or
“global” if they’re meant for everyone. Each service has read rights for
<service>-* and global-* parameters. We use a python library called SSM-cache
to cache values locally and avoid the cost of a look-up each time.

I’ve recently seen secret manager and I first thought it was a rebranding of
SSM PS. A quick google search seems to indicate that this is more or less the
case at the moment, but the service will probably pick up new features as it
matures.

~~~
gtsteve
There is no additional charge for SSM parameter store but rate limits apply
(and I can't find out what they are). I have read accounts from people on HN
who work on large AWS accounts who have been bitten by this.

The secret manager bills per request, so presumably that means they'll happily
serve as many requests as you want. It also has some nice tooling around
cycling secrets.

It's an important distinction when choosing which one to work with.

To contribute an additional idea not seen in this topic yet - my application
predates both these services so it just reads from a JSON file in an s3 bucket
that is encrypted at rest. It works just as well as all the above.

~~~
dmlittle
You should look into getParamtersByPath instead of getParameters. You should
be able to get past the rate limits for quite some time (we haven't hit the
getParamtersByPath limits yet)

------
dmlittle
Similar to laurentl we use AWS SSM parameter store in combination with
Chamber[1] to injects our secrets as environment variables to our application
code.

For services running on ECS, each task definition has an IAM Role attached
that has permissions to use secrets stored in parameter store with a prefix
unique for that service (each service encrypts its secrets with its own KMS
key). For example, serviceA will use the prefix serviceA and have a secret
under serviceA/DB_PASSWORD and serviceB will have its own secret under
serviceB/DB_PASSWORD. Each service cannot access each others secrets and even
if they could, they also need access to use the corresponding KMS key before
they're able to read the secrets. Since secrets are added as environment
variables we can inject them during startup with chamber. The entry point for
your docker container might look as following `CMD ["chamber", "exec",
"serviceA", "\--", "./executable"]`. Chamber will inject the secrets store in
SSM before starting the executable which reads the secrets from the env
variables.

For other services such as Lambda we do not use environment variables, instead
we use the AWS SDK to fetch secrets during cold starts and cache them for the
duration of the Lambda container. If you're using Node.js lambda functions
this is a little bit trickier to do than Go or python lambdas but it's still
possible. The Lambda function also has an IAM access to only fetch the secrets
under its own prefix.

For adding, removing or editing secrets we can use the chamber CLI with each
individual devs AWS access keys. For example: `chamber write serviceA PASSWORD
hunter2` to write a new secret or `chamber list serviceA` to list set secrets.

[1]
[https://github.com/segmentio/chamber](https://github.com/segmentio/chamber)

