
BeeGFS, the Parallel Cluster File System - randomname2
http://www.beegfs.com/content/
======
hkothari
Can someone explain how this differs from something like HDFS? I'm kind of
confused as to why there's no comparisons or mentions of HDFS here which makes
me think I'm missing something important about what this provides that's
special. Is it the easy to install part?

~~~
birc_a
HDFS is designed mostly for immutable files, stores three copies of everything
in the default setup and has the whole hadoop calculation stuff.

BeeGFS is closer to a normal NFS share. In the default setup it just stores
the data once, striped over multiple machines and it supports efficient
updates in-place etc.

------
leni536
[http://www.beegfs.com/docs/BeeGFS_EULA.txt](http://www.beegfs.com/docs/BeeGFS_EULA.txt)

Umm, how is this compatible with GPL v2? It seems more restrictive (enterprise
features, parts that would be better enforced by a trademark policy).

~~~
rbanffy
Speaking of trademarks, there is a band who would want to talk to these guys.

~~~
6502nerdface
How is there any potential for commercial confusion?

~~~
rbanffy
You could store music in the filesystem ;-)

------
poelzi
found this interesting:

[https://indico.cern.ch/event/346931/session/4/contribution/6...](https://indico.cern.ch/event/346931/session/4/contribution/62/attachments/684674/940478/beegfs-
at-desy.pdf)

> Plain BeeGFS is as reliable as your hardware

currently i would give MooseFS, orangefs or lizardfs a try.

Had experience with ceph and glusterfs and both I can not recommend.

~~~
pjc50
Could you elaborate on this please?

~~~
eis
I'd be also interested in a bit of info why Ceph and GlusterFS disappointed
you. I tried GlusterFS a few years ago and it took a good while to figure out
the right setup but in the end had disappointing small file performance.

Ceph was very interesting but wasn't product ready when I looked at it and
since then it somehow never got to a stage where people would rave about it so
that alone makes me think it's maybe too complicated. A bit like BTRFS which
also for years and years is being developed and sounds really promising but
never seems to be finally ready without gotchas.

~~~
mattbillenstein
I also tried ceph and gluster before settling on moosefs a couple years ago --
gluster was slow for filesystem operations on a lot of files and it would get
into a state where some files weren't replicated properly with seemingly no
problems with the network for physical servers.

Ceph at that time was unstable -- once I had a couple boxes kernel panic in
testing, I jumped off that boat immediately.

Moosefs is pragmatic -- written in pretty tight C, performant, and the web UI
for seeing the state of the cluster and how files are replicating is very
nice. We have a small cluster with maybe a dozen nodes running for almost 2
years now with no hiccups...

~~~
scurvy
Kernel panic? Were you using ceph or cephfs? Ceph block storage shouldn't have
any loaded kernel modules.

~~~
mattbillenstein
We were trying out the filesystem -- it was clearly alpha/beta quality at that
time, so it was no surprise it didn't work, and perhaps it's improved in the
last two years.

~~~
scurvy
cephfs is clearly marked as not for production. It still has more than its
fair share of bugs and has single points of failure. From the very top of the
docs page on ceph FS:

"Important CephFS currently lacks a robust ‘fsck’ check and repair function.
Please use caution when storing important data as the disaster recovery tools
are still under development. For more information about using CephFS today,
see CephFS for early adopters"

RedHat is focusing on making the block and object storage bulletproof. They're
content to let the opensource side twiddle away on ceph FS.

------
shiftpgdn
The company I work for has been running BeeGFS for 6 years with no issues.
Write and read speeds on Dell storage hardware match or exceed the NetApp
appliances we use.

We do occasionally have issues where ALL storage nodes must be rebooted but
other than it works great.

------
unwind
The only thing I wonder is if the name is a pun on
[https://en.wikipedia.org/wiki/Bee_Gees](https://en.wikipedia.org/wiki/Bee_Gees),
the band from the 1950s.

~~~
rvense
Allegedly it's good at Staying Alive...

------
merb
Actually there isn't a easy file system which spans 3 computers and scales to
thousand and is eay to install. There are only one's which are hard to
install, or needing more than 3 computers, etc..

~~~
Diederich
Honest question: how does GlusterFS not meet these goals? I know that it's
easy to install, and can easily run on three computers. And I believe it can
scale to thousands.

~~~
ollybee
Gluster provides files storage but not block storage.

~~~
Diederich
True enough, thanks!

------
craigyk
I will say that I know two people that have independently tried fairly large
trials (400TB+) of Lustre, GlusterFS, and BeeGFS, and BeeGFS was their
eventual favorite.

~~~
simonebrunozzi
Nice anecdote, but can you provide at least ANY detail on why they picked
BeeGFS?

------
sharktour
This doesn't fill the huge gap for Cloud NAS. GlusterFS is hugely complicated
and a fragile install, SoftNAS has (anecdotally) poor performance on high-
io... but it's closer. Amazon hasn't launched their EFS product to General
Availability Yet.

So wtf do we do when we need a 5TB shared storage platform across some EC2
nodes?

~~~
illumin8
Check out Avere systems. They have a clustered NFS appliance that you can spin
up in EC2, which translates back-end S3 object storage into NFS for your
clients. It supports between 3 and 50 nodes, and uses local RAM and SSD cache
to accelerate performance. Avere has also been in the NAS acceleration/caching
business since 2008, so their product is very mature.

~~~
sharktour
Thanks for the heads up on Avere. Use case for us was for file parsing as part
of a batch data pipeline. The latency we saw on alternatives was killer
compared to directly mounted EBS. That would be my fear with an S3 option as
well, especially when it presents like NFS.

------
rubyfan
Weird, the URL to the GPL'd source code behind a registration wall.

~~~
Loic
For what I read, this is to know a bit who is interested in getting the source
code. I give you the code but you tell me who you are. This is not prohibited
by the GPL.

~~~
pjc50
The obvious response to this silliness is for one person to download it and
then make it available by torrent.

~~~
DoubleMalt
Or mirror it on GitHub

~~~
sigio
The source is at:
[http://www.beegfs.com/source/](http://www.beegfs.com/source/)

------
mixmastamyk
It's a tragedy to think of how deep is your love for the project. Wonder how
long it will be stayin' alive? Gives me a night fever just thinking about it.

