
Think twice before encrypting your HFS+ volumes on High Sierra - tobiasrenger
https://bombich.com/blog/2017/09/29/think-twice-before-encrypting-your-hfs-volumes-on-high-sierra
======
mc32
A warning before FS conversion would be nice, given such a system isn't
mountable by any previous OS version. This might be an example of where
thorough QA seems to have gone down hill, if Apple Support think this behavior
was "impossible".

~~~
greglindahl
Great idea! That's exactly what the author of the article says at the bottom.

(Edit: reworded.)

~~~
nsxwolf
Why discuss anything in the article at all if we have responses like yours to
look forward to?

------
igammarays
Seriously regretting the upgrade. Am I the only one having random graphics
issues, such as rendering bugs, latency, and freezing, on a multi-monitor
setup on a Late 2013 MBP? Lesson learned: always wait a few months after a
major version bump.

~~~
balls187
Those issues have been fine with me, and my Late 2013 MBP.

VMWare Fusion + Bootcamp, however...

~~~
wila
For Fusion and Bootcamp, especially if you converted to APFS, you'll need
VMware Fusion 10. Then on top of that you'll have to remove the VM that points
to the bootcamp partition and recreate that bootcamp VM.

Some have reported success when using that scenario. Unfortunately not
everybody has been that lucky, so far it is unclear why.

~~~
balls187
I haven't converted my disk yet, but that's good to know.

~~~
wila
Keep an eye on this thread [0] at the VMware Fusion forum.

Seems that it depends on if you installed Windows 10 by BIOS or EFI. For BIOS
installed configurations you would have to disable SIP in order to use the
bootcamp partition as a VM.

Hopefully VMware finds a better solution.

[0]
[https://communities.vmware.com/thread/572924](https://communities.vmware.com/thread/572924)

------
Osiris
Another reason is that read/write speeds on APFS encrypted volumes are 30-40%
slower than non-encrypted volumes[1].

[1]
[https://news.ycombinator.com/item?id=15333036](https://news.ycombinator.com/item?id=15333036)

~~~
bouke
As the speed test was using a beta of High Sierra / APFS, I performed some
tests of my own. I've written /dev/zero to the disk, and read
Xcode_9_GM_seed.xip from the disk using different block sizes using `dd`. I
ran the read tests a few times, as there was a lot variation in the results.
I've excluded the first result of the read results for each FS. The test was
run on a Samsung Evo Pro 850 500GB SSD inside a Sharkoon USB-C case connected
over USB-C to a MacBook Pro 2016 w/ Touchbar.

    
    
        Speed in MB/s
                    HFS+    HFS+ Encrypted  APFS        APFS Encrypted
        1M write    498     473             490  (-2%)  444 (-6%)
        1M read     6896    6815            6860 (-1%)  6827 (0%)
        4k write    486     477             485  (-0%)  445 (-7%)
        4k read     2734    2709            2599 (-5%)  2545 (-6%)
    

While there is still some performance degradation between HFS+ and APFS, it's
within a single digit now.

~~~
stirner
dd is not an acceptable benchmark. You need to eliminate the effects of
buffering/caching to get accurate results.

~~~
Someone1234
/dev/zero is also not an acceptable source of data; since \0\0\0\0\etc is
unnaturally easy to compress. A large real file (e.g. installation ISO) may
have been a better idea.

------
kkylin
Also, AFAIK, if you use your disk for Time Machine backups, it cannot be APFS:
[https://support.apple.com/en-gb/guide/mac-help/disks-you-
can...](https://support.apple.com/en-gb/guide/mac-help/disks-you-can-use-with-
time-machine-mh15139)

------
to3m
Even if you stick with Sierra, precisely to avoid this sort of thing, you
might fall foul of the disappearing password hint problem:
[https://apple.stackexchange.com/questions/299558/](https://apple.stackexchange.com/questions/299558/)

------
jason_slack
So, I went to my local Apple Store because I could not format my boot volume
to APFS on my MBP-15(non touch-bar). I had backed up and already wiped my
drive. Clean slate.

The Apple Store genius flipped out and was like "why are you doing this?",
called me a "fan boy" and just was in general disbelief that somebody would be
upgrading a boot volume. He sort of dropped comments throughout our
interaction about just how bad an idea this was. It made me feel weird about
making the decision.

------
bgentry
> APFS volumes cannot be reverted to HFS

Is that actually true? I’d heard that Apple had reversion instructions for
those running the High Sierra beats on a system with a Fusion Drive, which
they dropped support for in the final release.

~~~
tzs
Converting between different file systems in place is an interesting technical
problem.

Let's say you have a disk that is formatted with filesystem S (for "Source"),
and you want it change it to filesystem D (for "Destination").

Let's assume for now that S is not encrypted. Let's also assume that we have a
fair amount of scratch storage available somewhere not on the disk we are
trying to convert. (I'll address removing this requirement later).

One approach is to make something that given a list of files, including
metadata, and a blank disk, can create a D filesystem on that disk and then
copies the files to it.

You also need something that given a file on S can give you a list of the
block numbers that hold that file's data.

That's an S to D copier, not an S to D in place converter, but it is useful
for testing your understanding of D. Once you are confident that you know how
D works, you can then make a new version of your D copier.

The new version doesn't actually write anything out to the blank disk.
Instead, it writes instructions for making the disk out to a file in scratch
space. For blocks that would hold metadata (e.g., inode blocks on Unix
filesystem), it records in the scratch space what their content would be and
where they would go on D.

For blocks that would hold file data, you write into the scratch area the list
of where that data resides on S, and where that data would go on D.

What you end up with, then, in the scratch space is essentially a set of block
by block instructions for copying S to a blank disk and ending up with a D
filesystem, where instructions are either "copy block X from S to block Y of
D" or "Fill block Y of D with the following literal data".

So to do an in place conversion, just follow those instructions, but with S
and D the same, right?

NO!!!!! That would likely totally trash things, because it is quite possible
that a source block in one instruction may have already been overwritten as a
destination in an earlier instruction.

What you first need is something that goes through the whole list of block
copy/fill instructions and figures out an order they can be done in that is
safe.

It is possible that there is no such order. In that case, you also need some
free space that can used for temporary storage. Given some free space you can
always find an order for the block copy/fill that works.

The free space can either be on another device, or if S is not full you can
find and use its free space.

Anyway, once you do all this, you end up with a simple list of data move/fill
steps that will convert S to D in place. You just have to figure out how to
actually execute them. The tricky case is when S is your system disk so you
cannot un-mount it. Your OS might get very unhappy if you start converting it
in place, and you also have to worry that it might write things to it in the
middle of the conversion that could corrupt it.

Probably best would be to do the conversion booted from something like a Linux
live CD.

I said earlier that you needed some separate scratch space to store the
information for the new filesystem. If there is free space on S you can
probably use that.

This comment is getting long, so I'm going to cut it off here. In a reply to
it I'll toss in a couple of implementation tricks that can be useful,
especially if D is a proprietary filesystem that you do not have a complete
spec for and so cannot make your thing that figures out where to write D
metadata and file data.

~~~
Someone
I think it is easier. Split your source disk in:

\- directory blocks that describe the directory structure and, for each file,
where to find this data on disk.

\- data blocks that contain actual file data.

\- free space.

Chances are you don’t need to touch the data blocks, and you have the free
space as scratch space. All you have to do is to write a new set of directory
blocks for the new disk format, and then delete the set of old directory
blocks. If the on-disk format is flexible enough w.r.t. the location of
directory and data blocks and you’re somewhat lucky, you can mostly write the
new set of directory blocks in existing free space on the source disk.

That would lead to an intermediate state where the source disk still has the
old format, but has one extra file that contains most of the new directory
structure.

The final, risky part then is fairly small: overwrite the disk’s root blocks
so that the system sees the disk as having the new file format with the new
directory structure.

If the disk formats use different disk blocks as first starting points, you
can even have a disk that can be used in either format (IIRC, that was
possible with FAT and HFS. FAT has its master directory block in sector 0, HFS
has it in sector 2)

~~~
tzs
Yeah, that would be a good approach if you have enough free space.

You could also have a preprocessing step where you rearrange files on the
source disk so that the free space is arranged optimally for holding the
directory structure for the new filesystem.

Speaking of rearranging files...another fun technical problem is doing a
(mostly) portable disk defragmenter. Given one filesystem/OS dependent
function,

    
    
      block_addr * blocks_for_file(char * pathname)
    

which given a file returns a list of the addresses of the disk block addresses
of the blocks that contain the file, and a filesystem/os dependent function
that can set the metadata (ownership, permissions, etc) for one file to be the
same as another file, you can write a defragmenter where everything else just
uses portable functions, such as C stdio. It won't be quite as good as a
defragmenter that knows filesystem internals and does low level writing
directly to the disk, because the (mostly) portable defragmenter cannot easily
move directory blocks around in a predictable way. Still, you can get most
files defragmented.

You can first build a map from files to disk blocks with the blocks_for_file
function. You can also map the free space by filling the free space with files
and using blocks_for_file to map those. When filling the free space to map it,
you should fill it with many one block files rather than a few large files,
for reasons that will soon become apparent.

Then suppose you want to move fragmented file X to blocks B, B+1, B+2, ...,
B+N-1, where N is the size of X in blocks. First you find all files that
overlap [B, B+N-1]. For each file Y that you need to move out of the way you
do as follows (remember, earlier we filled up all the free space with small
files, so right now the disk is full):

1\. Delete just enough of our free space filler files to free up enough space
for Y.

2\. Copy Y to Y.new, including copying ownership and permissions.

3\. Delete Y.

4\. Rename Y.new to Y

5\. Fill up the space that was freed in step #3 with one block files.

After you have done this with each file that overlapped [B, B+N-1], all of
those files now reside in the original free space, and [B, B+N-1] is covered
with small files you created. You can now delete those files, leaving the disk
with just enough free space to let you make a copy of X, which will end up in
that free space and be consecutive. You can then delete the original X, and
fill the newly free space with more one block files, and we are back where we
started, but with X defragmented.

Repeat for every other file you need to defragment.

What I like about this approach is that you defragmenter doesn't have to know
anything about how the underlying filesystem works.

Going the other way, defragmenting when you do know intimate details of the
filesystem, there is also a very cool albeit probably impractical approach to
defragmenting.

First, you figure out where everything currently is and where the free space
is.

Then, you have to peek at the filesystem's in-memory data structures to find
all the state that it keeps at runtime concerning allocation.

You need to know enough, and grab enough runtime data, that you can do all of
the following:

1\. Accurately predict where a file copy operation will place the new file,

2\. Accurately predict how that will change the internal state,

3\. Accurately predict how file deletion will change the internal state.

These all have to be good enough that given the state of the filesystem (on
disk and in memory), you can look at proposed series of copies and deletes,
and correctly predict what the final layout of the disk will be.

When you can do that, you can in theory then write a defragmenter that looks
at the state of the disk and the in-memory state, decides how it would like
the disk to be arranged, and then writes out a shell script that accomplishes
that entirely with a long list of cp, rm, chown, and chmod commands (or the
equivalent if you are not on a Unix or Unix-like system).

~~~
tzs
Kind of tangentially, the defragmenter stuff reminds me of another interesting
thing you can do given low level information about layout and access. I'm not
sure how much of the following is still applicable on modern filesystems and
hardware. It was pretty effective back at the end of the 20th century and
early 21st century.

Consider a program like Photoshop or a web browser. If you watch the I/O it
does while starting there are a lot of cases where it opens some file, reads a
few k, then goes and reads from a bunch of other files, and eventually comes
back and reads more from that first sale.

It often happens that the data it reads from that first file is actually
consecutive on the disk, but because it read it in two separate reads
separated by many reads from other files it has to do a seek when it comes
back for that second part.

This typically happens for many different files during a launch. Font files,
dynamic libraries, and databases, for example.

If you make a record the I/O sequences during many launches of a given program
you also find that they are mostly the same. There might be a few differences
due to it making temp files, or due to differences in the documents you are
opening on each launch, but there is also a lot of commonality.

At this point you can get clever. Make something that can tweak the I/O
requests made during application launch. When a program starts launching, your
tweak thingy can check to see if you have a log of a previous launch. If you
do it can load that and then for each I/O during the launch it can predict if
data beyond the extent of that particular read is also going to be needed. If
so, it can add another read to grab that data, reading it into a temp buffer
somewhere.

That might seem pointless, because the launching program is still going to
come back and try to do a read from that same part of the file later. Yes, it
will...but now the data from your early read might still be in the system's
file cache, saving a seek.

I'm simplifying a bit. What you would do in practice is analyze the logs of
the prior launches, identify which requests caused seeks, and then taking into
account the size of the system's file cache and whatever you know about how
the system cache works, figure out which reads should be extended and which
should not be (because they don't incur extra seeks, or because their data
won't hang around in the cache long enough).

Basically, you are preloading the cache based on your knowledge of what I/Os
will be upcoming.

On Windows 98 doing this could knock something like 30% of the launch time for
Microsoft Office programs, Netscape Navigator, and Photoshop.

I was curious once if this would work on Linux, probably around 2000 or so. I
made some logs of Netscape launching by simply using strace to record all the
opens and reads that occurred during a few launches.

I then identified several small files that had multiple reads during launch
separated by reads of other files, and then make a shell script that just did
something like this:

    
    
      cp file_1 > /dev/null
      cp file_2 > /dev/null
      ...
      cp file_N > /dev/null
      exec /path/to/netscape $*
    

where file_1, ..., file_N were some of the files that had multiple interleaved
reads during launch. I made no attempt to just read the parts that were needed
(which could have been done with dd) as I just wanted a quick test to see if
there was a hint that things could be sped up.

Launching netscape via my shell script turned out to be something like 10-15%
faster than normal, if I recall correctly. I was surprised at how well it
worked considering that doing it this way (do all the cache preloading up
front) should result in fewer cache hits than the "preload the cache for each
file the first time that file is accessed during launch" method.

~~~
Someone
Mac OS X did that in version ??? to speed up system booting. See
[http://osxbook.com/book/bonus/misc/optimizations/#ONE](http://osxbook.com/book/bonus/misc/optimizations/#ONE),
which claims it halved boot time.

------
alexlongterm
Don't think twice. Keep backups and encrypt your laptops if you care about any
of your data and your laptop travels beyond your desk and a locked safe.

------
j45
Glad I upgraded to Sierra just before High Sierra came out. Always good to be
a release behind, and upgrade to the latest if there's a reason to, or 6
months or a year after all these wrinkles are ironed out.

APFS looks very promising, it's hard to let Apple use me as a live beta
tester.

~~~
ellisv
> Always good to be a release behind

You at least patch your system right? Being behind did Equifax no favors.

~~~
sitharus
Apple maintain security updates for at least 2 previous revisions of macOS. As
long as you keep your system updated you’re as secure as the current version.

~~~
jesseendahl
There are unfortunately many security fixes that are not backported. A quick
Google search turns up numerous incidents:
[https://www.google.com/search?q=apple+security+kernel+not+ba...](https://www.google.com/search?q=apple+security+kernel+not+backported)

There are also numerous incidents mentioned by Google Peoject Zero, if you
read through their macOS disclosures.

It’s definitely unsafe from a security standpoint to run anything other than
the latest release.

~~~
j45
I wasn't aware of this.

Still, running something like little snitch, behind a router, along with some
browser plugins can go a long way. Until Sierra, I have never had an upgrade
that went smoothly, something critical I used has always broken.

Unfortunately Apple's OS updates are as bad as a zer0 day exploit in how they
can knock one's setup inoperable, sometimes for many weeks or months until
there's a way to fix, or revert.

Software vendors I'm using seemed to be out ahead of the High Sierra train
which was encouraging.

~~~
Shank
> Unfortunately Apple's OS updates are as bad as a zer0 day exploit in how
> they can knock one's setup inoperable, sometimes for many weeks or months
> until there's a way to fix, or revert.

You do backup things right? Disk cloning through Carbon Copy Cloner is super
fast and restores are painless.

~~~
j45
In my mind I don't own a macbook as much as I own a warranty and an
immediately bootable backup.

Apples backups were the reason I switched to Mac from windows a decade ago.
Can't be beat.

I run Time Machine on one hd at office, carbon copy cloner on another hd at
home, all they the USB hub.

It's not a backup if it doesn't exist in more than 1 place.

I was referring to the case where half your apps don't work and the other half
benefit greatly from the new version.

