ZFS but, tune it properly by limiting the size of ARC and buy some SSDs to use as both log devices and cache devices.
Also, do not buy any controller cards that will get in the way of ZFS talking directly to the disk (such as Dell PERC cards), where you see a "virtual" disk instead of the real one, it will affect performance.
Thank you. I have an old Optiplex as a number crunching toy and had just decided to use the two currently unused SAS drives to run a first ZFS test. You just saved me a lot of frustration.
Regarding the ARC, is zfs not as efficient about evicting the cache as, say, ext4? With ext4 I've never had to worry about it consuming as much memory as it pleases.
My layman's understanding: under Linux the ARC is not integrated into the usual Linux caches which are unified and which the kernel understands. So basically the ZFS ARC is a separate chunk of RAM, which can grow and grow (thus you need to limit it) and it won't be evicted as perfectly as the kernel would evict stuff cached on xfs or ext4 filesystem.
This is great for consistency and storage, but the first part somewhat sucks if you've built it into your layer as well (like hadoop fs). And it makes writes & reads a tiny bit slower than it should be.
They have fat xattr inodes, which is great if you use it - but useless if something like SELinux means nothing to you.
The point where they truly excel is in a multi-disk scenario.
And in that land, the zpool array implementation makes it awesome to introduce an SSD+HD mixture with nearly the best of both systems.
BTRFS doesn't have that yet (well, bcache on the lower layer, perhaps).
But as someone who's fired up debugfs a bit too many times, if you really don't know which one you need, pick the one that ships with your distro!
> And in that land, the zpool array implementation makes it awesome to introduce an SSD+HD mixture with nearly the best of both systems.
With an array of half SSDs and half HDs, can I configure ZFS so that I never hit the IOPS bottleneck of the HDs? How do you recommend configuring the array?
I have never used a setup like that - all I've done is throw an SSD in front of 3 disks.
That said, the zfs log and zfs cache command supports mirror disks for log and allows more than 1 cache per pool.
Using more than 1 SSD as a cache (and of course, using mirrored logs for safety) would be recommended if you had a production box with this.
To repeat, I have never tried ZFS-on-Linux anywhere near a production setup - but if I had to, I would use multiple SSDs just in case one locks up its firmware.
Both have good features, both will be useful and interesting in a few years.
Right now I'd not trust either of them, in production use. Even if you never spent 60 minutes restoring backups both filesystems are too new for me to trust, with my conservative tendencies.
Also, do not buy any controller cards that will get in the way of ZFS talking directly to the disk (such as Dell PERC cards), where you see a "virtual" disk instead of the real one, it will affect performance.