

Ask HN: Why don't cloud file stores like Amazon S3 have query capability? - hoodoof

So you have an Amazon S3 bucket with millions of items in it.<p>Seems logical there would be some sort of query language to identify files according to a range of criteria.<p>I wonder why there&#x27;s no way to do so?
======
eldavido
There's no way to do so because that's not how the system is built. Queries
require indexes, which require more I/O, some level of storage consistency
across the system (so that the query can run against a single "snap" of the
database rather than something that's constantly changing!), and a lot of
other things that just aren't designed in S3.

A common pattern is to store bulk data in s3 but metadata (filenames, sizes,
etc.) in a database for faster queries. If you need the ability to make
arbitrary queries on the data itself, S3 isn't the product to use for this.

Data architecture is a fundamental part of your system; don't fuss over
spending some time on it!

------
wmf
Because that would be a database and would have database prices.

~~~
hoodoof
I wouldn't mind that, it's not having any way to do queries that is
frustrating.

~~~
nl
Then you want to run ElasticMapReduce[1] against your S3 buckets.

[1]
[http://aws.amazon.com/elasticmapreduce/](http://aws.amazon.com/elasticmapreduce/)

