
ZFS: Apple’s New Filesystem That Wasn’t - swills
http://dtrace.org/blogs/ahl/2016/06/15/apple_and_zfs/
======
tracker1
It's kind of a shame that ZFS hasn't seen better adoption, and that btrfs has
been pretty stagnant, and that in general next generation file systems are
mostly still-born. It would be nice to see some improvements in this area as
10tb hdd's are eminent, and data management is important.

However, it's worth noting that RAID, especially in software on systems
without ECC ram is less than ideal. Beyond this is the overhead of managing
larger filesystems with ZFS. The DIY raid servers that support it have had
some troublesome issues that I've experienced first hand.

It's likely a lot of these advantages have been displaced by the
discontinuation of Apple's server projects as well as other fears. By similar
note, I've always been somewhat surprised that NTFS hasn't been more popular
for external media, as it's a pretty good fit there.

In the end, software at this level has been held back significantly by the
patent hornets nest that exists in the world today. I truly hope that we can
see some reform in this space, but with the likes of TPP and similar treaty
negotiations around the world today, that is pretty unlikely. Eventually some
countries will have to strike out, break these bad treaties and reign in IP
law. The vast majority of software is undeserving of patent protection. Just
as software really shouldn't have copyright law that lasts for decades. It's a
shame all around.

~~~
technion

       NTFS
    

I ran into a bug just today that I'm still struggling to believe. A large
Oracle server fell over, just reporting "IO errors". The issue was ultimately
traced to NTFS's handling of fragmentation[0].

I've gone about ten years thinking running a defrag - particular on a virtual
disk sitting on a SAN in a RAID array - made no sense. Today's downtime says
otherwise.

It's one thing to perform poorly. But seeing services just die with disk write
errors on a disk that's 30% full has left me with a very low opinion of NTFS.
Our Oracle person sounds like a desktop maintenance person from the 90's
making an issue out of "the importance of weekly defrags".

[0] [https://support.microsoft.com/en-
au/kb/967351](https://support.microsoft.com/en-au/kb/967351)

~~~
Spooky23
In my DBA days in the 90s/2000s, this sort of issue was usually a lousy DBA
that had extent sizes set too small.

Windows can clearly handle big databases. The people getting paid to run the
platform need to have a clue about the platforms they are running.

------
rdtsc
In general is it just me or has ZFS become more popular lately. Saw Ubuntu get
behind it, even in light of Btrfs being available for many years now.
[https://github.com/zfsonlinux/zfs/commits/master](https://github.com/zfsonlinux/zfs/commits/master)
is pretty active...

It seems everyone at some point expected Btrfs to shoot ahead and leave other
file systems in the dust, so there was no point in bothering with ZFS, "just
wait a bit and Btrfs will be the default everywhere". And besides ZFS has all
the legal issues with it.

But it seems Btrfs progress was rather slow, so even in spite of legal issue
interest in ZFS is still growing.

~~~
barkingcat
the thing about ZFS is that it was an actual selling/functioning product
before btrfs was in development (links at end). There always were people using
it - whether through the main Sun/Oracle Solaris product or via one of the
OpenSolaris / illumos / Freebsd / Bsd continuations.

Btrfs was not an fs in wide production use the way xfs or ext3 / ext4 were, so
it's understandable that btrfs progress is slow. The two pieces of software
are just really far apart in timelines and maturity. People are paying
attention to ZFS mainly because it's had a history of working and functioning
in production environments and in heavy load, for a decade now.

In my opinion, in order to have parity with ZFS, some big distribution (bigger
than Suse, maybe Debian or Ubuntu) needs to set btrfs as its default fs, and
have banks, stock exchanges, dns providers, virtual hosting companies, etc
hammer the hell out of it for another 6 or 7 years and get all the bugs that
come out of those experiences fixed. Then people will start paying attention
to it.

afaik btrfs development started 2007
([https://en.wikipedia.org/wiki/Btrfs](https://en.wikipedia.org/wiki/Btrfs))

ZFS was introduced as a part of Solaris in 2005, released as openZFS in 2006
([https://en.wikipedia.org/wiki/ZFS](https://en.wikipedia.org/wiki/ZFS))

~~~
Twirrim
Honestly... I don't trust btrfs at the moment. It has had some nasty data
losing bugs in it in the past, and admittedly its been a while since I heard
of one, but it's also not often that you'll hear of someone actually using it
in production. Even their wiki page is woefully lacking in examples with which
to sell it:
[https://btrfs.wiki.kernel.org/index.php/Production_Users](https://btrfs.wiki.kernel.org/index.php/Production_Users).

I want to like it, we really need a nice, stable next-generation file system
in Linux land (that doesn't have the licensing issues associated with ZFS)

~~~
mjevans
I do like it. However I am very wary of using the advanced features (beyond
the de-fragmentation/scrubbing/balance kind) without a /good/ backup of the
data on the system and a good retention policy.

It has been my experience with using BTRFS for my own personal data that it's
the less tested code paths which had / might have the issues you're
describing. Things like sparse files (I got bit by this bug), snapshots
(regular scrubbing and backups are things you're doing already right), and
other features that are less frequently used.

A small production cluster where I work uses XFS over BTRFS though, because of
the very FUD that's mentioned and the type of storage happening on it matching
the sparse pattern (even though that bug should be fixed).

------
jhugg
Seems like running on SSD/NVRAM may call for some new thinking. Running on
watches may be even secondary to that.

Not that ZFS won’t run well on SSD, but it feels like there’s a gulf between
filesystems designed with SSDs in mind and those designed with spindles in
mind.

~~~
tracker1
It seems that COW filesystems in general are better for use with SSDs... iirc,
SSDs themselves do similar things as part of their wear leveling algorithms
anyway. It seems to me that it's time to break some of this down. With solid
state storage reaching levels close to RAM in terms of latency, I think
eventually they'll come together, and that's where it will get really
interesting.

On the flip side, with spinning storage approaching 10tb drives, the need for
enhanced filesystems for RAID arrays with those beasts are needed as well.

In general, we really need some patent releases from those companies
strangling filesystem advancements (namely Oracle and MS).

~~~
jlgaddis
Out of curioisity, how do you feel that Oracle is strangling filesystem
advancement?

~~~
dkh
Encryption is a big one that's been annoying everyone. Oracle baked encryption
into their own ZFS implementation ages ago, but not in a way that works with
the open source variety. Years later, we're only just now starting to see some
solutions for this.

------
raattgift
Meanwhile, this works, is under active development, and is essentially at HEAD
of both openzfs and zfsonlinux:

[https://openzfsonosx.org/](https://openzfsonosx.org/)

~~~
Analemma_
Not sure what your point is. Are you saying Apple should use this? AFAICT it's
still under the CDDL, which is the main reason Apple abandoned the ZFS plans
in the first place.

~~~
cylo
Except for the fact DTrace is under the exact same license and ships with OS
X.

~~~
zepto
DTrace is not a core technology on which the rest of the platform is built. If
you read the actual article, this is discussed.

~~~
ahl
Agreed with all of this, but it's hard to imagine the licensing was truly
insurmountable.

~~~
msbarnett
More or less everyone I've ever talked to or seen discuss it agrees (including
several people in the comments on this blog post) -- Apple Legal felt ZFS was
a no-go without patent indemnification from Sun/Oracle. Sun was willing, but
Oracle said no as soon as they purchased.

At that point the word came down from Apple Legal to Engineering that it
wasn't happening. And there's nothing more insurmountable than your own legal
department.

~~~
DannyBee
"At that point the word came down from Apple Legal to Engineering that it
wasn't happening. And there's nothing more insurmountable than your own legal
department. "

Just to note: It doesn't really work this way, even at Apple.

It really means "legal was better at arguing their side than whoever argued
against them to the SVP/CEO who made the decision"

While apple is worse than most (from talking to friends and counterparts), no
company, even Apple, is so silly that the legal department can't be overruled
(any general purpose software company that works like that goes bankrupt
pretty quickly).

It just takes a really good business case for doing so, and none existed here.

That is,if it had been considered super-mission-critical, it just would have
happened anyway, they would have taken the risk.

(I kind of just hate when these stories get portrayed as legal ruling with an
iron fist with nobody having any say over them.)

------
cbsmith
People forget that Microsoft has perennially had their Cairo/Object/Whatever
FileSystem in development.

~~~
usefulcat
True but NTFS is also far superior to HPFS, so arguably MSFT doesn't need a
replacement for NTFS nearly as badly as Apple needs a replacement for HPFS.

~~~
stormbrew
Minor nit, but HPFS was OS/2's filesystem. Apple's is HFS (and HFS+).

~~~
Zardoz84
And NTFS was based on HPFS

------
gribbly
So, according to this article, Apple was making a deal with Sun to use ZFS,
and later finally dropped it after an alleged discussion between Steve Jobs
and Larry Ellison when Oracle owned it.

My question is why did Apple think they needed to make a deal to use ZFS in
the first place, and if so has Canonical (who says they'll ship openZFS with
Ubuntu) made a deal with Oracle ?

It's true that OpenZFS is more than Oracle's ZFS, but unless I'm mistaken, the
vast majority of code in that project is still owned by Oracle.

This article makes me uneasy.

~~~
ahl
Apple started in 2006 with no deal or coordination with Sun. Later when the
FUD started, they were looking for protection from patent lawsuits (i.e. from
NetApp), for support and expertise, and, possibly, reassurances around patent
clauses within the CDDL with which I'm not an expert.

------
snarfy
Maybe I'm being irrational, but I would never touch ZFS because Oracle.

------
falcolas
> this is the moral equivalent of using newspaper as insulation: it’s fine
> until the completely anticipated calamity destroys everything you hold dear.

How... unnecessarily inflammatory. Hardlinked backups work remarkably well,
and are incredibly simple to implement and understand. Of course they can be
corrupted, but then again, so can every other form of backup in existence
(that said, there are no protections built-in to a hardlinked backup).

~~~
feelix
I work in backup (this has become a core part of my business:
[http://macdaddy.io/mac-backup-software/](http://macdaddy.io/mac-backup-
software/)) and part of the reason I wrote that is because of the continual
reports of Time Machine backups failing to restore when people needed them.
The numbers are huge. I wouldn't touch Time Machine in any conditions. I would
manually backup before using that. This may be anecdotal evidence, but if you
want to see some of the issues people run into with Time Machine just read
through some of the posts people make about it:
[https://www.google.com/search?q=time+machine+backup+failure&...](https://www.google.com/search?q=time+machine+backup+failure&gws_rd=cr,ssl&ei=lddhV6zQL8G7mwX3u5OABg)

Most of this is indeed directly related to directory backups. I also do
Snapshots of sorts, but it only makes hardlinks to files, not directories.
That proves to be remarkably more reliable.

~~~
Kluny
Hey, I'm interested in trying that. What's your business model? Nagware or
limited time trial or what? The price is reasonable but I'd definitely like a
long trial.

Anyone else here using it?

~~~
feelix
It gives a month's free trial where it fully functions. Then it says it
requires payment when it starts up after that. Though if you don't pay
anything it actually continues working. So you have plenty of time to try it
out.

------
beedogs
It's a real shame, because HFS is an absolute abomination. Apple will remain
without a reliable, full-featured filesystem for another decade, it seems.

------
williesleg
ZFS is fantastic until you try to grow a volume. Emphasis on fantastic.

~~~
kinghajj
What difficulty did you encounter? I've had a pool running since ~2009,
originally as 4x750GB, then 4x3TB in 2013, and I recall that the transition
was excellent. As oofabz said, the trick is to enable "autoexpand", which can
be done even after creation. In fact, I recall that I had replaced each drive,
initially seeing no increase in capacity, but after enabling autoexpand the
new space was immediately available.

~~~
Sidnicious
You can grow a RAID-Z “vertically” like this, but can't grow it horizontally
by adding disks… which is frustrating.

(I'm also frustrated that it's not possible to change redundancy within a vdev
— e.g. go from RAID-Z1 to RAID-Z2.)

~~~
oofabz
You can grow it horizontally by adding a new vdev. ZFS doesn't allow you to do
anything you may ever possibly want to do, but it does allow you to do more
than any other RAID solution. It's not perfect but it's still pretty
impressive.

