
Apple’s new APFS file system - robin_reala
https://mondaynote.com/apples-new-file-system-who-cares-3ba440cd4d84
======
bluejekyll
> the “iOS First” release once again speaks to Apple’s priorities, to its
> vision of its future

I actually disagree about the reasoning behind this strategy. iOS may be more
important to Apple at the moment, in terms of market value, but there's a
better reason to target it first. iOS is deployed to constrained systems,
iPhone, iPad, appleTV... this is easier to test because there will be less
variance to deal with than on macOS deployments. It makes a ton of sense to
start there for technical reasons so that they can validate it in more
consistent environments.

~~~
eric_h
I'd also like to mention that it is extremely impressive that Apple was able
to transparently swap out the filesystem, with no data loss, on millions of
devices. I've not heard of anyone who had serious issues with the update. It
was an incredible technical achievement.

~~~
linkmotif
Yeah that's amazing.

More generally, Apple was providing working OS upgrades for years while the
notion of upgrading Windows between major versions was basically laughable.
Wasn't Windows 8 the first version of Windows that you could actually upgrade
to and have the system boot successfully? Maybe Vista to 7 worked for people.
I never bothered trying. But I remember upgrading from OS X 10.3 to 10.4 and
being floored when everything just worked.

~~~
swang
i'm really confused about this.

i have upgraded (or my dad had anyways) from win 3.11 to win 95 and win 95 to
98. i think i clean installed win xp when i got a new computer but eventually
i upgraded that to win 7.

all of them ran fine. or was that just not the norm?

~~~
acdha
Windows 3.x, 95, 98 and ME all used the DOS FAT filesystem (or the extended
FAT32 variant). Windows XP supported both FAT and NTFS so if you did a clean
install you were probably using NTFS.

What iOS 10.3 did was basically the equivalent of having a Windows upgrade
which converted FAT/FAT32 to NTFS in-place without the user having to manage
the process.

~~~
kirb
Windows XP did actually include a built in FAT32 to NTFS upgrade tool. Of
course, you did have to run it manually from a command line, but I never had a
failure from it. Vista would convert automatically if it saw a FAT boot drive,
and I think Windows 2000 would upgrade FAT16 to 32.

I wonder if Apple will take the same approach for macOS (convert manually if
you want for now, or we'll use it if you reinstall). Seems like a far greater
risk for Apple if upgrading the OS breaks something for a lot of people, when
they want users to be completely comfortable about updates.

~~~
acdha
I was trying to remember if CONVERT was added in W2k or XP but if memory
served you couldn't run it on the boot partition.

~~~
kirb
Seems it was 2000:
[http://www.cs.toronto.edu/~simon/howto/win2kcommands.html](http://www.cs.toronto.edu/~simon/howto/win2kcommands.html)

It definitely worked on the boot partition at least on XP, not sure about 2k.
If it can’t lock the drive it schedules it to be done at boot time (similar to
on-boot chkdsk).

------
rubatuga
Addressing data integrity is a slippery slope of comparing hard drive bit read
errors and ECC ram bit flips. The reality is, for things like video and
pictures it probably won't be a problem. However, if the data is sensitive,
you should be using a raid 6 with 2 disk redundancy, coupled with constant
data scrubbing to ensure that the disks remain bit rot free. The probability
of bit rot occurring at the same place is probably less than 1 in 10^16,
possibly even less if data scrubbing is constant.

Here's a good article addressing bit rot:
[https://lockss.org/locksswiki/files/ACM2010.pdf](https://lockss.org/locksswiki/files/ACM2010.pdf)

~~~
Tanegashima
People keep forgetting that this FS was designed by Apple for use in their
__appliances __.

Apple devices, save for their least sold model (MacPro), in their least sold
product line (Mac) are consumer devices with no ECC RAM. Most people don't
know what ECC RAM is, most don't care, and won't pay more for it.

For Apple, it doesn't make sense to switch to ECC for system memory, however,
if they implement storage ECC at block level in their SSD controller (Apple
designes their own SSD controllers), they can obtain the same functionality at
much lower cost (only flash controller RAM needs to be ECC).

~~~
scott_karana
> Apple designes their own SSD controllers

Huh! I assumed they used off the shelf stuff. Do you have any articles about
this? :)

~~~
mikhailt
The first time I've heard of this was when Anandtech did their review of
iPhone 7 and it has the Apple's custom SSD controller:
[http://www.anandtech.com/show/10685/the-iphone-7-and-
iphone-...](http://www.anandtech.com/show/10685/the-iphone-7-and-
iphone-7-plus-review/4)

They mentioned the '15 MacBook was the first to use Apple's custom SSD
controller, which iFixit took a picture of here:
[https://www.ifixit.com/Teardown/Retina+Macbook+2015+Teardown...](https://www.ifixit.com/Teardown/Retina+Macbook+2015+Teardown/39841#s90315)

They most likely started off from Apple's purchase of Anobit, the custom flash
controller designer, back in 2011:
[https://en.wikipedia.org/wiki/Anobit](https://en.wikipedia.org/wiki/Anobit)

------
agildehaus
Unfortunately bit rot is not something APFS addresses as it only checksums
metadata. Wonder if this is something Apple can add to APFS in the future?

~~~
shouldbworking
I hope. It wouldn't be such a big issue if Intel stopped stripping ECC from
desktop CPU's. Traditionally bad ram causes more bit rot than damaged drives.
Ram get less reliable every die shrink, everything should be ECC by now.

~~~
zamalek
Why not just go with AMD? ECC is supported on all Ryzen SKUs.

~~~
vetinari
But that does not mean that all the boards support it, even if the chip works
at all. What good ECC RAM does, if the errors are not logged?

------
bensummers
APFS doesn't do proper checksumming on data, and won't mirror or RAID, so it
doesn't have the data recovery properties implied by the article.

[http://dtrace.org/blogs/ahl/2016/06/19/apfs-
part5/](http://dtrace.org/blogs/ahl/2016/06/19/apfs-part5/)

~~~
hyperknot
Isn't it possible that Apple developed this filesystem primarily to be used
with their SSD based devices which are using their custom controllers, which
simply takes care of this low level protection? SSD controllers are said to be
very complicated piece of proprietary technologies, and I believe they are
using sophisticated algorithms for integrity protection.

~~~
vetinari
Apple users use external (USB, Thunderbolt) drives too, where they would not
get this special hypothetical protection.

~~~
justinjlynn
Clearly telegraphing Apple's intent to release new "Apple approved (and
licensed)" storage "accessories" for their systems.

~~~
tedd4u
Maybe, but isn't the recent trend at Apple to discontinue desktop-only
accessories (like monitors)?

[https://9to5mac.com/2016/11/21/apple-reportedly-stops-
develo...](https://9to5mac.com/2016/11/21/apple-reportedly-stops-developing-
airport-wifi-router-products-moving-employees-to-other-projects/)

[https://www.theverge.com/2016/6/23/12020510/apple-
thunderbol...](https://www.theverge.com/2016/6/23/12020510/apple-thunderbolt-
display-discontinued)

~~~
Analemma_
At the recent Mac Pro announcement, they also said there would be a new Apple-
branded monitor, so they're not out of that game completely.

------
comex
Nitpick: You can, in fact, create new folders in the iCloud Drive app on iOS.
For some reason it's hidden under the Select option.

------
pmarreck
For those who don't know of him, Jean-Louis Gassée (the author here) is a
great. Check out some of his quotes:
[http://www.azquotes.com/author/38975-Jean_Louis_Gassee](http://www.azquotes.com/author/38975-Jean_Louis_Gassee)
my favorite probably being
[http://www.azquotes.com/quote/754436](http://www.azquotes.com/quote/754436)
about Apple Computer.

He also invented the legendary BeOS.

~~~
luckydude
Isn't the file system guy from BeOS the guy behind the apple fs? Yeah, he's
referenced it's Dominic Giampaolo. Not sure how much coding he did but it says
he's one of the architects.

BeFS was way way ahead of its time. Really a shame that OS didn't get picked
up by more people.

~~~
savoytruffle
I think he's also credited as the father of the add-on filesystem journaling
that was enabled on consumer OS X 10.2.2 back in fall 2002, and many of the
shims and patches that have kept HFS+ going to this day.

------
hackuser
The author links to a much more technically informative article:

[https://www.backblaze.com/blog/apfs-apple-file-
system/](https://www.backblaze.com/blog/apfs-apple-file-system/)

------
mark_l_watson
He is joking with the title. Money quote: "encryption will be easier to use,
disk space will be better utilized, backups will be more reliable"

Both my iPhone 7 and my iPad Pro only have 32G of storage and the new file
system gave me back extra storage due to better utilization. It was
interesting how smoothly the transition went.

------
anton_gogolev
Bit rot is _absolutely_ not about SSD degradation.

~~~
hoschicz
How did the backup software detect it then? Or is the author falsely calling
SSD degradation bit-rot

~~~
cmurf
Hard to say without dmesg, but it could be metadata corruption and in the
ensuing confusion backupd gives up; or it could be an explicit read error
report by the drive which would result in an I/O error to backupd and it'd
likewise give up.

------
spilk
The most interesting part of this article (for me at least) are those early
Mac screenshots - The Flounder? and that next one with not-quite-
Chicago/Geneva fonts in the Finder? Where did those come from? I realize JLG
had been with Apple since the early days of Mac development but I've never
seen anything like those before.

~~~
joezydeco
Jean-Louis kind of speaks incorrectly here when he says _" Very old-timers
will remember the pre-Mac File system called The Flounder"_, unless by old-
timers he means Apple employees that were on the Mac team.

There's a little more about the early Mac software history at Folklore.org,
including another screen shot of what eventually became Finder:

[http://www.folklore.org/StoryView.py?project=Macintosh&story...](http://www.folklore.org/StoryView.py?project=Macintosh&story=Early_Demos.txt)

[http://www.folklore.org/StoryView.py?project=Macintosh&story...](http://www.folklore.org/StoryView.py?project=Macintosh&story=Busy_Being_Born,_Part_2.txt)

~~~
rbanffy
Funny... I converted Cream 12 for use in Apple II programs.

For the old-timers, it was a shape table and multiple draw commands of
numbered (ascii value - 32) shapes would allow you to write proportionally
spaced text to the graphical screen. It was the same format Take-1 used (we
used their programmer's toolkit).

So, Cream 12 came down from Altos all the way to our Apple II+'s.

------
HappyTypist
What's the best way to protect myself against bit rot, as someone with
extremely slow upload speeds?

~~~
boomboomsubban
The best way is using ZFS with parity.

------
n1000
Since there seem to be experts here in the discussion: I keep thinking about a
little toy open-source project to adress bitrot in TimeMachine backups. A
utility that compares chekcksums of the original file against its backup.
Apparently Apple started using hashes internally in Time Machine and it would
probably be simplest to use those. However, I cannot find any documentation on
the internals of Time Machine. Does anyone know where to start looking? Would
such a tool make sense at all?

~~~
RJIb8RBYxzAMX9u
tmutil verifychecksums should already do this.

~~~
n1000
As far as I understand it, this will verify checksums within the time machine
backup by comparing hashes with the ones that were created during backup.
However, it will not compare the files against the ones on the system. So it
protecte against bitrot on the backup disk only.

~~~
RJIb8RBYxzAMX9u
Well, you'll need some kind of heuristic to figure out if a mismatch is due to
bit-rot, or because the working file's changed. Maybe modification date, but
a) what if a file's restored from an older backup, b) mtime itself is
corrupted (APFS would guard against this I suppose).

In any case, the "hash" appears to be CRC32, stored in extended attributes:

    
    
      $ xattr .inputrc
      com.apple.finder.copy.source.checksum#N
      com.apple.metadata:_kTimeMachineNewestSnapshot
      com.apple.metadata:_kTimeMachineOldestSnapshot
      $ xattr -px 'com.apple.finder.copy.source.checksum#N' .inputrc
      26 E5 4A AB
      $ cksum .inputrc
      2873812262 65 .inputrc
      $ printf '%x\n' "$(cksum .inputrc | cut -d ' ' -f 1)"
      ab4ae526

------
robertlagrant
What on earth is nanosecond data resolution? Does it just means file
create/modify timestamps are saved with that level of precision?

If so, who cares about that?

~~~
aembleton
Apparently it's important for tracking the order of operations. I don't know
why a ledger couldn't be kept.

[https://arstechnica.com/apple/2016/06/digging-into-the-
dev-d...](https://arstechnica.com/apple/2016/06/digging-into-the-dev-
documentation-for-apfs-apples-new-file-system/)

~~~
moomin
If it's anything like systems I'm familiar with, it is a ledger: if two things
happen at exactly the same time, one happens a nanosecond later.

------
mehrdada
Worth noting that iOS HFS+ already had some non-standard features [hacks] that
Mac did not use like encrypting files with different keys based on the data
protection level—that APFS provides —so it makes sense to upgrade iOS first,
which can readily leverage such functionality, and it not necessarily a direct
result of it being the "vision for the future".

------
Exuma
What is the release date for APFS on MacOS?

------
dingo_bat
> A director of _the largest PC OS licensing company_ once told me they spent
> more money on driver development and fixes than on OS code proper.

Funny way to refer to Microsoft.

~~~
newjersey
Isn't this true of gnu/Linux as well though? A lot of patches are drivers and
fixes? For example, one could argue trim for solid state storage is a driver
issue?

~~~
dingo_bat
Of course it is. Drivers will be the majority of the work in any OS that is
not locked onto a very limited set of hardware.

------
TonyHou
16GB iPhone users care!

------
rs86
This article is so boring I considered killing myself

------
eridius
Can we please change the title? The current one sounds like an inflammatory
anti-Apple article, and the only reason I even clicked it was in expectation
of a good laugh.

~~~
rubatuga
It's a rule not to change the titles when posting, unless there's a very good
reason not to do so

~~~
teach
The current guideline is "please use the original title, unless it is
misleading or linkbait."

I probably would have left the original title alone, personally, but you could
make a case that it was misleading. HN moderators apparently agreed and have
since changed it.

