
Hundreds of exposed Amazon cloud backups found leaking sensitive data - ewood
https://techcrunch.com/2019/08/09/aws-ebs-cloud-backups-leak
======
joncrane
I've been working almost exclusively in the AWS space for about 10 years now.
Clients anywhere from tiny little three-person consultancies to Fortune 100.
Commercial, govcloud, dozens of clients.

Never once have I ever found a use case for making public EBS snapshots.

Who on Earth is thinking that it is a good idea to take an EBS snapshot and
make it public?

Note, several of those engagements did involve multiple accounts, and the need
to share / copy AMIs and/or snapshots between accounts. But never making them
public.

~~~
dboreham
Laziness in attempting to share data with someone in another org?

"Nope, can't access it" ...

"Nope, still can't access it"...

"My manager is harassing me to get access now"...

"Look, just make it public then change it back after I get it copied"...

~~~
lowdose
The guy that produces that last line definitely wears a suit.

~~~
atoav
As much as I dislike the typical suit wearer, the industry as a whole would be
much better if IT-people wouldn’t blame everything on them and just tell them
you won’t do it. If you are a welder and you boss wants you to weld a gas tank
that hasn’t been emptied you tell him that he has no idea what the thing he
asks for means, explain why and wait till the gas tank is emptied and
everything is checked.

I worked as a Camera guy in film and I am known to be very fast – yet I had
directors who wanted things even faster. Bit there is a natural limit to how
fast you can get something done without having worthless garbage as a result.

You can take certain risks, skip certain advisable steps, focus on the most
essential thing etc, but below that there waits literally nothing.

In my experience taking a step back, breathing in, out, and then proceed to
doing it properly is in most cases faster than following your boss into panic
and ditching common sense.

It is _your_ responsibility as a professional to say “No” or “Stop” in certain
circumstances. And if they _really_ want you to do it, write down the possible
consequences of what they force you to do and make them sign that they take
the full responsibility.

There is so much talk about IT security with people frowning upon silly
behaviour, yet any other craft would bend over backwards before you could
force them into unsafe behaviour. If engineers would build bridges like we in
IT operate, we would have many collapses a day.

~~~
tialaramex
(Not advice to parent, just generally)

If you want to be more professional about this stuff, build up a Fuck Off
Fund. Women in particular have written about fuck off funds in the context of
making sure you don't have to nod along when HR says the VP who stuffed his
hand down your top was "just fooling around" but - most people need that
financial security any time they have to confront the boss. Save to be able to
look the big boss in the face and tell them "Fuck Off". "Fuck Off" isn't the
response you need when they tell you they want the database authentication
disabled "just until Monday, Tuesday at the latest" that's when you want "No".
But you need to know you _can_ tell them to "Fuck Off" so you actually say
"No". Otherwise you may find yourself agreeing anyway.

~~~
yourapostasy
For the youngsters out there, don't elaborate when you say "No". That is,
don't mention your FOF. That weakens your position.

It is sufficient to look the manager/executive threatening you square in the
eye, and state your position with deliberateness. Keep it professional, no
raised voices, and be willing to walk away without hesitation if the other
side gets abusive.

The really bad ones are those who tell you that if you step through that door
don't bother coming back or similar, so be absolutely ready to commit. If you
do the main event behind closed doors one-on-one and get that threat as you
walk out, sometimes they'll come back with sugar suggesting a do-over.
Generally Admiral Ackbar is right in this context, but it's your call.

The negotiation leverage that comes from the FOF is most powerfully
communicated non-verbally and in a face-to-face setting, also through body
language. The difference is very noticeable between those who have an FOF and
those who don't, if you have enough experience. It can be faked, but it takes
exceptional practice to fake. The tell starts with how fast and confident the
"No" comes back.

This is the nuclear option of course. Exhaust all other avenues of reasonable
negotiation first. Like an email with witnesses you pick for deliberately
violating departmental policy, for example.

------
jedberg
The creator of the first Ubuntu distros for EC2 wrote about the dangers of
public EBS snapshots 10 years ago:

[https://alestic.com/2009/09/ec2-public-ebs-
danger/](https://alestic.com/2009/09/ec2-public-ebs-danger/)

He just got notified by AWS a couple days ago about the public snapshot he
mentioned in the article.

But at least AWS is trying to make things better here by proactively checking
for public EBS snapshots and notifying people.

------
d2mw
Public EBS snapshots are great, and thankfully a design other clouds didn't
copy. I've found all kinds of stuff in there, including a 900GB Oracle backup
of a publicly traded manufacturer's accounting system. It doesn't require much
imagination to understand how this kind of data could be profited from, given
relatively low effort

It seems unlikely a lot of people didn't already know about this, it's hard to
miss even if you only spend a few days with the EC2 API, and it's also quite
surprising AWS have yet to correct the design. 90% chance it is mostly a UI
problem -- there are no warning labels around snapshotting in the EC2 UI

~~~
rolltiide
How do you scour for EBS snapshots and open browsable S3 buckets?

~~~
d2mw
For EBS, step 1 is reading the docs, step 2 is cutpasting a documentation
example.

For S3 I'm not sure how people are building their lists. AFAIK the API
provides no enumeration. So this is possibly something coming from web crawl
data (e.g. common crawl)

~~~
moduspol
Perhaps something like this?

[https://github.com/eth0izzle/bucket-
stream](https://github.com/eth0izzle/bucket-stream)

------
cfstras
Oh god, how much I hate articles that don‘t list their sources. Where are the
slides from?

The talk description is here:
[https://www.defcon.org/html/defcon-27/dc-27-speakers.html#Mo...](https://www.defcon.org/html/defcon-27/dc-27-speakers.html#Morris)

------
kpeekhn
AWS Trusted Advisor has warned of this since 2017:

[https://aws.amazon.com/about-aws/whats-new/2017/06/aws-
trust...](https://aws.amazon.com/about-aws/whats-new/2017/06/aws-trusted-
advisor-now-checks-for-public-snapshots-of-amazon-elastic-block-store-ebs-and-
amazon-relational-database-service-rds-data/)

------
AaronFriel
I just checked an EC2 console and I can see 19,356 snapshots created by other
users.

I am so confused.

It would be trivial to make finding a snapshot require knowing a unique ID
like an AMI.

And, why do I need to be able to search for 1000s of customers' public
snapshots in the EC2 console? What conceivable purpose does that serve except
being a giant opsec fail?

~~~
judge2020
looks like some bigger images were of wiki[pedia|media] backups so I guess
easy-to-find/use public data?

------
snazz
It’s still true that most security issues are caused by human ineptitude, not
clever vulnerability-hunting or burning sophisticated zero-days.

~~~
eli
I would replace "human ineptitude" with "flawed system design that makes it
very easy to make very bad mistakes"

~~~
dragonwriter
But is the flawed system design here the automated system at AWS or the human-
in-the-loop systems by which companies are providing admin access to IT
resources, including AWS accounts?

~~~
eli
Probably both! But I would argue that below a certain size provisioning things
by hand probably makes sense. A UI that makes it too easy to make a private
thing public is never ok.

~~~
dragonwriter
Sure, but I don't think the current EC2 UI makes it too easy (and the S3 UI
could only make it harder by not making public and cross account access
possible at all.)

------
chimen
Gotta appreciate the hijack of the back button on techcrunch. Bounce rate too
big?

------
jcims
FWIW same thing is possible with RDS db snapshots and dbcluster snapshots.

------
andrewstuart
I had a simple glance in the console and there are like 20,000 exposed ebs
snapshots - available for anyone to copy and examine - I think that's only for
a single region too - switch regions to see more.

Amazon should make an emergency decision to make all these private.

Sure it will break stuff but I'd be disappointed if Amazon left what is in
effect a security hole open for the sake of backwards compatibility.

They should also give me a single click link when I sign in to show me all of
my public ebs snapshots and throw it hard in my face when I sign in to the
console so I simply cannot avoid seeing them all.

I have multiple AWS accounts and I just signed in to try to see if I have any
public EBS snapshots and then I realised I would need to search __every single
region in every single account and then select every snapshot one by one __to
find out. That 's a huge problem. I need a single click to show me every
exposed snapshot across every region in my account.

UPDATE:

I can't say for sure if this is 100% right but I think if you sign in to your
AWS account, then click on each of these links, you will find if you have
public snapshots.

Maybe someone else could confirm if this is correct?

[https://us-east-1.console.aws.amazon.com/ec2/v2/home?region=...](https://us-
east-1.console.aws.amazon.com/ec2/v2/home?region=us-
east-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://us-east-2.console.aws.amazon.com/ec2/v2/home?region=...](https://us-
east-2.console.aws.amazon.com/ec2/v2/home?region=us-
east-2#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://us-west-1.console.aws.amazon.com/ec2/v2/home?region=...](https://us-
west-1.console.aws.amazon.com/ec2/v2/home?region=us-
west-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://us-west-2.console.aws.amazon.com/ec2/v2/home?region=...](https://us-
west-2.console.aws.amazon.com/ec2/v2/home?region=us-
west-2#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ca-central-1.console.aws.amazon.com/ec2/v2/home?regi...](https://ca-
central-1.console.aws.amazon.com/ec2/v2/home?region=ca-
central-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://eu-central-1.console.aws.amazon.com/ec2/v2/home?regi...](https://eu-
central-1.console.aws.amazon.com/ec2/v2/home?region=eu-
central-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://eu-west-1.console.aws.amazon.com/ec2/v2/home?region=...](https://eu-
west-1.console.aws.amazon.com/ec2/v2/home?region=eu-
west-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://eu-west-2.console.aws.amazon.com/ec2/v2/home?region=...](https://eu-
west-2.console.aws.amazon.com/ec2/v2/home?region=eu-
west-2#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://eu-west-3.console.aws.amazon.com/ec2/v2/home?region=...](https://eu-
west-3.console.aws.amazon.com/ec2/v2/home?region=eu-
west-3#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://eu-north-1.console.aws.amazon.com/ec2/v2/home?region...](https://eu-
north-1.console.aws.amazon.com/ec2/v2/home?region=eu-
north-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-east-1.console.aws.amazon.com/ec2/v2/home?region=...](https://ap-
east-1.console.aws.amazon.com/ec2/v2/home?region=ap-
east-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-northeast-1.console.aws.amazon.com/ec2/v2/home?re...](https://ap-
northeast-1.console.aws.amazon.com/ec2/v2/home?region=ap-
northeast-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-northeast-2.console.aws.amazon.com/ec2/v2/home?re...](https://ap-
northeast-2.console.aws.amazon.com/ec2/v2/home?region=ap-
northeast-2#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-northeast-3.console.aws.amazon.com/ec2/v2/home?re...](https://ap-
northeast-3.console.aws.amazon.com/ec2/v2/home?region=ap-
northeast-3#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-southeast-1.console.aws.amazon.com/ec2/v2/home?re...](https://ap-
southeast-1.console.aws.amazon.com/ec2/v2/home?region=ap-
southeast-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-southeast-2.console.aws.amazon.com/ec2/v2/home?re...](https://ap-
southeast-2.console.aws.amazon.com/ec2/v2/home?region=ap-
southeast-2#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://ap-south-1.console.aws.amazon.com/ec2/v2/home?region...](https://ap-
south-1.console.aws.amazon.com/ec2/v2/home?region=ap-
south-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://me-south-1.console.aws.amazon.com/ec2/v2/home?region...](https://me-
south-1.console.aws.amazon.com/ec2/v2/home?region=me-
south-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

[https://sa-east-1.console.aws.amazon.com/ec2/v2/home?region=...](https://sa-
east-1.console.aws.amazon.com/ec2/v2/home?region=sa-
east-1#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

~~~
judge2020
non-region console links should redirect OK if you're signed in -
[https://console.aws.amazon.com/ec2/v2/home#Snapshots:visibil...](https://console.aws.amazon.com/ec2/v2/home#Snapshots:visibility=public;ownerAlias=self;sort=desc:startTime)

------
chovy
[https://fullstacknews.com/t/devops](https://fullstacknews.com/t/devops)

