* Lost/stolen hard disks from AWS datacenter
* Attacker or rogue AWS employee who has access to low level storage but not keys.
The article is correct that this misses many important situations but it is still better than nothing.
Another often overlooked benefit of at-rest encryption is that it can enable crypto-shredding; delete the object key and the object can no longer be recovered. That's faster, more efficient, and better for the environment compared to other kinds of "secure erasure". Of course if you want robust crypto-shredding you do have to take some care about where the keys might be cached and for how long, but you can't do it without encryption at-rest.
The feature is there for paperwork reasons.
Think about your risk model: there are a lot of employees at your cloud provider, and a lot of high-profile customers. There's a defense in depth approach here. Metal detectors, security guards, and searches to enter & exit data centers. Data encrypted when on-disk. Keep the keys in some KMS system. Physically isolate the KMS system, use HSMs, restrict permissions to the KMS.
Without these measures, you might end up with a system where ordinary customer support representatives, software engineers, data center technicians, or system administrators / SREs can exfiltrate customer data on a whim. At a large cloud provider like Amazon, Google, or Microsoft, that's not a small attack surface. That's a lot of people. The idea is to make it extremely hard for one of these employees to exfiltrate customer data without somehow leaving a record of it which ultimately gets the employee fired and prosecuted.
Customers will make mistakes too--screw up their permissions, fail to encrypt their data, or whatever. The idea behind defense in depth is that any one security measure can probably be circumvented, but multiple independent security measures are harder to circumvent.
It's really easy to be cynical about security but I have worked at cloud providers and the threats are real.
A combination of the default policy being poor on all AWS services, the shared responsibility model of AWS delegating this risk to clients and the default security posture in the industry being terrible makes the real problem of solving this like traipsing across a field covered in poo.
An analogy; you have to tick the box to put your front door key under the doormat. The burglar knows where the hole in your fence is and knows you might keep your keys under the door mat. Plus your kids left their keys on the github bus.
Of course the SOC2 audit says you ticked the box and you tell everyone your house is a castle.
I still put my seat-belt on, even though there's an airbag in the car.
I am not really sure its better than nothing - what it does do imo, is muddy the waters a bit for casual users, and perhaps make it seem like the customer doesn't really need to do as much securing themselves - because it is now 'encrypted by default' - giving a false sense of security - and as others have pointed out, really just protects against extremely rare edge case problems, i.e. the lost hard disk.
I am curious if cloud providers such as AWS provide default access to data stored in their data centers to the governments (to all data, and by default, in a successor to the PRISM program, not on a case by case basis and under subpoenas which is a different program)?
If so, KMS doesn’t help, and client-side encryption is the only way to protect the data.
I would only use client side encryption/decryption if I had really sensitive data and I truly couldn’t trust AWS with it.
The main advantage is more control over the key & permissions on your side of the shared-responsibility model.
AWS default key has a fixed policy and is wide open to your account, so it's not suitable for:
- Using different keys for different situations/data within the account; this can help with defense-in-depth & errors in least privilege from escalating. Example: Lambda accidentally has overly-permissive S3 permissions, and you're using the default key. It could potentially read more S3 buckets/files than intended. With different keys, it couldn't decrypt the data in other buckets.
- Using different accounts. Default keys are only usable within your account, you can't grant access to use it to other accounts (which might sound odd, but there's use cases where you'd consider doing this)
- Default keys can't be used by some AWS services, e.g. the default key policy can't be used by CloudWatch Events to enqueue to an encrypted SQS queue, for instance. (workaround is adding a resource-policy to the queue, but I personally do not like resource-based policies -- I like all my permissions in IAM roles & policies so there's one place to audit and control)
All this said, I often leverage the default keys heavily, as using your own KMS key is more expensive.
There are two ways to grant access to a KMS key: as a policy grant on a user/role, and as a policy on the KMS key itself.
If you don't have access to the key, then reading the S3 object doesn't work.
So I have to trust that AWS's security here is that the S3 API itself can't unless granted that access.
I'm still not convinced. Just seem like theater. S3 has its own access control. If you don't have access to the S3 object then reading the S3 object doesn't work ... Requiring additional access to the key seems redundant - theater.
P.S. I'm not arguing against encryption at rest. I'll take it, but using my own key doesn't seem to offer too much over using one of AWS's rel low level access.
For a case like this, the upshot of it is that with permissions on the KMS key, S3 is effectively locked out and can't fetch it. KMS is a hermetic system with no operator access with HSMs at its root, so that provides a pretty meaningful difference. It also means that controls can be handled by different customer teams, and that a single change on a shared CMK can render an unbounded number of S3 objects inaccessible quickly.
> KMS is a hermetic system with no operator access with HSMs at its root, so that provides a pretty meaningful difference.
So, if I don't use a CMK (Customer Managed Key), what am I getting instead? Assumed (perhaps naively) KMS would still be used under the hood.
> a single change on a shared CMK can render an unbounded number of S3 objects inaccessible quickly.
I feel like you could achieve that with an IAM policy. I guess the key can apply to a wide range of unrelated AWS services so act like a ~meta policy.
And yep, a CMK can span many services and still give you a single point of control.
But compliance is still the major thing. Many regulators, auditors and compliance authorities are just happier with KMS being the point of control. Even though AWS operates KMS, and the services, KMS is a neat compartmentalized hermetic system with HSMs at its root. So it's quicker and neater to evaluate its surface area, TCB, and so on and it's a familiar pattern to finance and health regulators especially.
For instance, if someone does a wildcard resource "acme-prod-widget-*" for a new policy, but forgets that widget also has (say) a customer-pii bucket, too.
Well, if acme-prod-widget-customer-pii is encrypted with a customer-pii specific KMS key, then there's no accidental leak of that data.
I know AWS has fixed this now, but in years past we paid a ton of money in KMS requests from s3 for these types of configurations and we asked ourselves what is this really buying us?
At the end of the day I have to assume some AWS employees have access to some or all keys in KMS.
KMS also has the capability to support an external trust store (https://aws.amazon.com/about-aws/whats-new/2022/11/aws-kms-e...) ... where AWS holds no key material at all.
If I’m right, S3 sends encrypted data encryption keys to KMS, and KMS sends back the decrypted data encryption keys. So, although S3 has no access to the master key, it has access to the data keys in RAM for the customer to use. With a “bucket key,” it goes further storing those type of keys in disk.
The key is stored in memory while it's being used to encrypt/decrypt, that's unavoidable, but humans don't have access to that. Bucket keys are a bit different, where there's a per-bucket key which has a similar scheme and then per-object keys are derived from it as needed. Together with random nonces/IVs, it ends up being a bit of a mini multi-party scheme.
It's incredibly easy to dumpster dive old hardware from datacenters (or waste management facilities). Even if their policy is to demagnetize and shred every hard-drive out of every dead machine, that's assuming they always follow their policy correctly. Humans are flawed (and subject to bribes).
I have friends who run K8s clusters on hardware they took from the dumpsters behind datacenters. (Not Amazon datacenters, but ones where the government has an entire floor dedicated to them)
Since I had to write the AWS API from scratch anyway (fringe language, and we tried to avoid C memory problems), I also implemented transparent encryption at the same time. (Using an off-the-shelf vetted implementation of appropriate encryption algorithms. And in such a way that, when handling large data, it could run on other cores if no accelerator.)
Of course, as the article points out, the EC2 instances (or whatever needs to access the data within the objects in S3) by definition need access, but at least that access is restricted more closely to what actually needs it.
I am wondering if AWS has access to data encrypted with the SSE-CMK keys as well?
They state that the SSE-CMK keys reside in tamper-proof FIPS-validated HSMs and not even Amazon has access to them. But it’s a bit misleading because they use envelope encryption. Furthermore, the encryption and decryption are still server side.
They could have better phrased it that, the only advantage of the SSE-CMK keys over SSE-KMS keys is that, with former, the users can define key policies (providing another access control layer similar to IAM) and monitor usage.
They might as well have open SSH access on all EC2 instances using root + “password”
What you're describing makes sense for a service like EBS where a single customer's bytes are limited to small number of disks in well-defined locations.
I would expect any mid to senior level engineer to understand the implications and add further security (local encryption before upload etc.) measures as neeeded.
This layer of server side encryption is designed to protect against an attack where someone breaks into an AWS data center, pulls a disk drive out of a storage system that is hosting S3 objects, and then tries to read the data off of that disk drive. Without the key (which is obviously stored separately, on a separate system) the disk will be useless to them.