
Small files are killing your query performance. Make bigger files instead - polistro
https://www.upsolver.com/blog/small-file-problem-hdfs-s3#utm_source=hackernews&utm_medium=referral
======
zimpenfish
A similar problem was faced by INN which stored each article in its own file
(like Maildir) - life got pretty crappy on big transit boxes even with
directory hashing. Diablo solved it by allocating huge files and storing
articles in there sequentially with a pointer stored elsewhere (like an mbox
file.)

