
Quantcast File System on S3 - lumens
https://www.quantcast.com/blog/quantcast-file-system-on-amazon-s3/?source=hn
======
lumens
For some context:

Quantcast moved off Hadoop in favor of a home-rolled solution (QFS) because of
Hadoop wasn't scaling well to our needs (40PB+ data processed/day).

This release of QFS
([https://github.com/quantcast/qfs](https://github.com/quantcast/qfs))
includes support for S3, among other improvements.

~~~
bertan
I wonder how it compares/relates to [https://minio.io](https://minio.io)

Can it run on minio, for example?

~~~
mck35
One difference I see is that QFS is block-based and each file is composed of
multiple S3 objects. Hence, there is no file size limit.

------
mck35
Hi, we recently announced Quantcast File System version 1.2. Besides many bug
fixes and improvements, it comes with the ability to use Amazon S3 as a
backend store. You can check it out on QFS Github. I personally contributed to
this release, and I'd be happy to answer any questions. Thanks.

