Got the idea from an AWS talk here https://youtu.be/7Px5g6wLW2A. Blew my mind. I coded up all the changes in a day. Took way longer to move the data than to code it.
AWS provides a ton of building block primitives you can use to build with at a price point better than you can do it yourself. If you just try to do it yourself using their IaaS offerings 24/7 (ec2, vpc, etc) then you're in for a bad time.
Let's be specific on speed. Most of the time, the MB or two needed for a plot is a fraction of a second, which our customers can deal with. That retrieval is a single object from S3, the way we organize things.
Having said that, the talk I linked to has some great advice - use the "folder" structure to write data so you don't search, you just use the naming scheme to do a direct object read. In addition, we can keep most reads to a single object, which is fairly fast.
As is always true, you will need to test to see if it fits your speed needs. But even with just the naming scheme and meta data in the database to eliminate all searches on S3, the speed works for us.