I'm looking forward to BcacheFS. ZFS on Linux is great when it works well, but it's an absolute pain when it breaks. Not only does it taint the kernel, but it doesn't really mesh very well in the kernel due to the usage of the SPL -- a layer used to convert Linux APIs to Solaris Kernel APIs. In addition, ZFS doesn't use as much native Linux memory management as I'd like, instead it manages its own pool of memory. This makes troubleshooting more difficult. This mechanism is further aggravated with the use of the kmem cgroup.
For example, if you have a dirty page in a cgroup, and the cgroup OOMs, the kernel will trigger writes. If any of these writes require memory allocations, they'll probably fail since the current cgroup is OOM. ZFS subsequently gets stuck in an infinite loop, and locks up. See: https://github.com/zfsonlinux/zfs/issues/5535
I understand that a lot of ZFS works comes from LLNL & government funding. I'm not blaming them, as it works for their use case of machines that are running dedicated, controlled workloads.
We're experimenting with Btrfs, and we'll see how it goes.
For example, if you have a dirty page in a cgroup, and the cgroup OOMs, the kernel will trigger writes. If any of these writes require memory allocations, they'll probably fail since the current cgroup is OOM. ZFS subsequently gets stuck in an infinite loop, and locks up. See: https://github.com/zfsonlinux/zfs/issues/5535
I understand that a lot of ZFS works comes from LLNL & government funding. I'm not blaming them, as it works for their use case of machines that are running dedicated, controlled workloads.
We're experimenting with Btrfs, and we'll see how it goes.