On: AWS Snowmobile – Massive Exabyte-Scale Data Transfer Service

[RangerScience]

Who even has that much data, outside of Google and Facebook?

I mean - I totally believe there are people that do have that much, I just don't know who they are.

Fake Edit: Film?

[itsmeduncan → RangerScience]

There is an example in the blog post about DigitalGlobe.

https://aws.amazon.com/blogs/aws/aws-snowmobile-move-exabyte...

[dzdt → RangerScience]

Any retail chain has that scale of data in surveillance tapes. Order of few mb/s * 10's of cameras per location * 100's of locations * 1000's of seconds per day * 100's of days per year => order of 10^15 bytes per year. So far it hasn't been economical to collect and analyze these tapes. But it is approaching feasibility, and AI is reaching the point it could give useful output on that massive input. Patterns of how shoppers interact with merchandise. What can you change to sell a few percent more items?

[Cyph0n → RangerScience]

File sharing services. Megaupload back in the day for instance, or services like Dropbox, OneDrive, and Box.

[halleym → RangerScience]

Banks?

[anotheryou → halleym]

That's just numbers and a bit of text, no?

I guess anything media related is way bigger. I thought of media archives of TV stations, but those are (if digital already) cold storage with only a low-res preview live.

[halleym → anotheryou]

Check images, receipts, transactions, statements (particularly PDF), deposit slips, mortgage documents... all kept for 7-20+ years.

[RangerScience → halleym]

Oooh. Very good point.