

Distributed FS: MogileFS - astine
http://www.danga.com/mogilefs/

======
jbellis
Mogile relies on a central mysql db for metadata (e.g. replica locations) so
it doesn't scale well at all, especially not when your workload emphasizes
many small files ( because then you exhaust mysql that much faster relative to
your disk capacity).

~~~
recurser
We're running about 90 terabytes of mogile storage containing 20-30 million
files and haven't had any problems at all. If we do, the plan is to shard them
along the same lines as our database sharding. I don't know of any other
products apart from S3 that could have made our file hosting as stress free as
it is. What kind of scaling problems have you had?

~~~
jbellis
so your average file size is 4.5MB. that's a pretty good fit.

> I don't know of any other products apart from S3 that could have made our
> file hosting as stress free as it is.

I think you're right about that part. Better solutions don't come off the
shelf. Yet. :)

> What kind of scaling problems have you had?

the system I wrote at Mozy needed to scale to PBs of space and billions of
files, so it was obvious that the mysql part would be a problem.

more generally I think mogile is in a pretty narrow niche: typically either
you really need a _lot_ of space and files, like flickr or facebook or ..., in
which case mogilefs doesn't cut it, or you have a smaller data set but you
want random access to your files in which case mogilefs also doesn't cut it.

~~~
recurser
Ha sorry I didn't realize who I was talking to, yeah if you built mozy then
mogile would probably hit the wall fairly early :) I work for one of the
larger Japanese web companies, and the CEO has basically declared war on
amazon japan (literally - they have anti-amazon tshirts, slogans etc) so S3
was not an option. Otherwise, a lot of people in this space would probably
find S3 a perfect fit.

------
jobenjo
We've been using MogileFS for the last few years to host all our avatars. It's
fast and powerful, and has had very few issues.

It's definitely a solid system, though was certainly overkill for our needs
(if I could do it again, I would just use s3).

