
Playing with ZFS encryption on Linux - funkaster
https://www.rolando.cl/blog/2017/06/13/playing-with-zfs-encryption/
======
binwiederhier
A little more details on the crypto aspects:
[https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-
zf...](https://blog.heckel.xyz/2017/01/08/zfs-encryption-openzfs-zfs-on-
linux/) (This is my post. I hope it's okay to post this here.)

~~~
heftysync
Have the design decisions been reviewed by a cryptographer yet? According to
the presentation on youtube, it had not.

~~~
ryao
We are having trouble finding a volunteer.

~~~
heftysync
Would the Core Infrastructure Initiative be willing to sponsor a cryptographer
to review it?

Getting this right would make it a lot easier to have faith in encrypted
backups. Right now, you have to trust that each network backup product like
bacula is doing it right.

I just randomly picked bacula and then looked up what they are doing. They're
using OpenSSL with AES-CBC. I don't think anyone would recommend that.

As a start, it would be nice if Datto could document the design decisions. For
instance, AES-CCM is a bit of a strange default and it's not explained
anywhere. I see Solaris chose AES-128-CCM and maybe that influenced why
OpenZFS is using AES-256-CCM. I haven't found a cryptographer in favor of CCM
over GCM. For instance, Matthew Green doesn't appear to like it over GCM.

[https://blog.cryptographyengineering.com/2012/05/19/how-
to-c...](https://blog.cryptographyengineering.com/2012/05/19/how-to-choose-
authenticated-encryption/)

According to Crypto++'s benchmark, AES-GCM is 2,448 MB/s and AES-CCM is 710
MB/s. Does that match your experience with OpenZFS?

[https://www.cryptopp.com/benchmarks.html](https://www.cryptopp.com/benchmarks.html)

Given that GCM IV reuse is catastrophic and one of the main use cases for this
is backups to potentially untrusted sources, I'm curious if AES-GCM-SIV would
be a more conservative choice.

[https://www.imperialviolet.org/2017/05/14/aesgcmsiv.html](https://www.imperialviolet.org/2017/05/14/aesgcmsiv.html)

~~~
tcaputi
I'm Tom Caputi, author of the ZFS encryption patch and I can answer of few of
your questions.

First of all, the choice of AES-CCM. I have had a few people ask me why we
didn't chose something like ChaCha20 as a block cipher instead of AES. This is
largely because AES is by far the most scrutinized block cipher around. It's
use is currently so widely accepted that modern Intel CPUs have built-in AES
instructions to improve performance. While its true to say that ChaCha20 (or
other block ciphers) might theoretically be faster or that they ARE faster on
some architectures like 32-bit cell phone CPUs, this is not currently the case
with the vast majority of ZFS deployments.

As far as the choice for CCM as a default goes, this one was a little bit
harder. Originally this decision was made to match the Oracle implementation
as much as possible (a design decision which has since been dropped). Later,
when we re-evaluated the descision, we found a paper
([http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/comment...](http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/comments/CWC-
GCM/gcm-update.pdf)) indicating there might be weaknesses with the
authentication mechanism, although the paper only mentioned cases with
truncated authentication tags. So in the interest of being as conservative as
possible, we chose the option which looked the most secure. We did not look
into AES-GCM-SIV since it is very new (it looks like it actually came out this
year) and so I would not by any means consider it a "conservative" choice.

As far as performance goes, we have not yet (as far as I'm aware anyway) seen
a case where read or write speed wasn't bottlenecked by the disk speed. The
benchmarks you posted are (as far as I can tell) single threaded and ZFS
processes each block asynchronously.

The biggest thing here is that AES-256-CCM is only the default. It is easy for
users to pick GCM for the time being and for developers to add newer, better
encryption algorithms and change the default as time goes on. I wouldn't be
surprised i we had changed the default by the time that the patch ends up in a
tagged release.

------
DCKing
I almost cannot wait for this. Native file system encryption will be so good,
because it removes the ugly/inelegant layer of indirection with Geli or LUKS,
and it allows stuff like encrypted ZFS send and receive. It allows you to
switch on and off encryption dynamically. That will allow you to use services
like rsync.net without having to place as much trust their internal security
practices.

It's great they're taking their time, but I've been looking forward to a
stable release since its announcement almost a year ago :)

~~~
cmurf
Uncertain that it enables encrypted send and receive. It very likely decrypts
to get it into the send format, and then is reencrypted on the receive side.
Those are separate file systems so it's not certain that they share, or even
should share, the same encryption key.

Update: Ahh well it helps to whole article before posting. Both encrypted and
decrypted send/receive are possible. But this also suggests multiple keys per
pool, a key per file system I guess?

~~~
tcaputi
Yes, there are per filesystem / zvol keys and the raw encrypted data is sent
over the wire along with the encrypted keys.

------
mavhc
Can I zfs send encrypted incremental snapshots to a remote server and have
them merged without ever having the remote server decrypt anything? That would
be awesome to backup to an untrusted remote computer.

~~~
binwiederhier
Yes.

~~~
mafro
Without having read anything of the implementation.. That is little short of
astounding.

------
aidenn0
64 bytes seems like an arbitrarily short limit for passphrases; it's going to
be hashed at some point anyway (preferably with a decent KDF designed for the
purpose) and 64 bytes is nowhere near long enough for natural English language
text to be used to derive 32 bytes of entropy (estimates of English language
text are ~1.5bits of entropy per character).

~~~
RJIb8RBYxzAMX9u
At the risk of invoking Cunningham's Law, _nobody_ uses passphrases anywhere
near 64 bytes that he / she has to memorize. And if you're using a password
manager, just use the raw or hex key option.

32 bytes => 64 hex digits, which is probably not a coincidence. :-)

~~~
takeda
Password yes, passphprase it's quite possible.

~~~
takeda
Eh, Materialistic doesn't allow me to edit my comment. I mean you're right
about passwords, but a passphrases (e.g. a sentences) are easy to remember,
more secure and can often be longer than 64 bytes.

------
FullyFunctional
Will this work the same way on FreeBSD/FreeNAS when this eventually lands?

OT: one of the sed lines is hard to read

    
    
       sed -i 's/github.com\/zfsonlinux\/zfs.git/github.com\/tcaputi\/zfs.git/' PKGBUILD
    
    

the OP might leverage that you can use other quoting characters, like

    
    
       sed -i 's,github.com/zfsonlinux/zfs.git,github.com/tcaputi/zfs.git,' PKGBUILD

~~~
Amezarak
FreeNAS has supported zfs encryption for a long while, unless I misunderstand
you. Unless its not actually zfs encryption but something else?

~~~
gruturo
Free(BSD|NAS) so far create a "normal" ZFS on top of an encrypted block
device, produced via the cryptographic GEOM provider geli (
[https://www.freebsd.org/cgi/man.cgi?geli(8)](https://www.freebsd.org/cgi/man.cgi?geli\(8\))
)

This instead is ZFS doing the actual encryption on a normal block device.

------
eslaught
Does anyone know a good guide for getting set up with ZFS (particularly on
Linux/Ubuntu)? The Ubuntu wiki basically just say "apt install zfs" and then
links to a page that appears to be a brain dump of all the possible commands.
Something with more coherence explanation of the relevant concepts would be
really helpful.

~~~
weitzj
I have documented the steps I had to take here
[https://janweitz.de/article/creating-a-zfs-zroot-
raid-10-on-...](https://janweitz.de/article/creating-a-zfs-zroot-raid-10-on-
ubuntu-16.04/)

Be aware:

Uefi did not work for me in VirtualBox with ZFS.

Booting with UEFI, ZFS and RAIDz works, but I did not figure out how to
install the UEFI grub-boot on all RAIDz drives.

Therefore, if the one and only drive with the boot partition fails, I have to
rely on an USB-stick as a backup-plan for the bootloader.

Probably if I could start over I would try to skip UEFI and use legacy booting
and installing Grub in the MBR of each drive (if that is possible).

For encryption we are using Luks right now, since it encrypts block devices
and ZFS uses these block devices to form its raid.

Luks needs to decrypt ALL raid devices inside the Grub boot prompt in order
for ZFS to start its raidz.

To make this happen, we had to modify:

/usr/share/initramfs-tools/hooks/cryptroot and remove an early return in a
for-loop in get_fs_devices() , since otherwise only the first device was
decrypted.

Make sure /etc/lvm/lvm.conf contains use_lvmetad = 0

Make sure /etc/default/grub contains GRUB_ENABLE_CRYPTODISK=y
GRUB_CMDLINE_LINUX_DEFAULT=“text"

So, yeah, I am looking forward for zfs-native encryption on Linux to make
booting "safer/easier".

~~~
vetinari
Maybe I had a better luck, for me, UEFI boot with ZFS in Virtualbox did work.
I've used CentOS though.

Multiple UEFI boot partition are possible, but just like you note in your
article, they have to be outside the ZFS pool. All of them. Then you set order
for each one using efibootmgr. For each, you will get a separate entry in UEFI
boot manager.

It is not necessary to install grub2 after rebooting into ZFS root. It is
perfectly fine to run grub2-install in chroot from your temporary
installation.

------
ubercow
I'm not really familiar with the development process between ZFS on Linux and
ZFS on BSD, do these pull requests usually get merged upstream for use on BSD,
Solaris, etc?

~~~
AndrewDavis
Together FreeBSD, ZoL and the Illumos groups work on OpenZFS.

Oracle closed off Solaris. The decendent of Open Solaris is Illumos. Oracle
ZFS and OpenZFS are not hte same thing.

OpenZFS repo is the upstream, though it is closely tracks the Illumos version.

From there the projects pull on OpenZFS to create their implementations. New
features are developed independently by the projects with rules that a new
feature must be enabled in a downstream distro for X period of time before it
can be upstreamed into OpenZFS.

~~~
ryao
Let's not forget Mac OS X and OSv, although the amount of work being done that
and shared is not as large as it is with the other 3 platforms.

------
XorNot
I've been really looking forward to OpenZFS encryption support. ZFS on LUKS
leaves a lot to be desired, and I've only just recently begun trying to shift
all my PCs and devices to be fully encrypted by default.

------
symlinkk
nice post, I think you forgot to fill in the Introduction though

~~~
funkaster
thanks! I think I stupidly lost that paragraph in a merge. I'll remove it for
now and add it later. Thanks again!

------
bfrog
I wish ZFS were in tree :-)

