
Sanoid – Replication and Backups with ZFS - mkj
https://github.com/jimsalterjrs/sanoid/
======
rsync
This is the piece I was looking for:

"Which would push-replicate the specified ZFS filesystem from the local host
to remotehost over an SSH tunnel ..."

    
    
        syncoid root@remotehost:data/images/vm backup/images/vm
    

Now, if only there were a cloud storage provider that you could directly zfs-
send to ...

If only ...

~~~
jimmcslim
I have an Ubuntu server with a ZFS pool that I want to rebuild as a
FreeNAS/TrueNAS server. Can I use something like rsync.net to push my datasets
up as a backup, rebuild my server, and then pull them back down into a
reorganised/renamed tree of datasets? Are there compatibility issues between
ZFS on Linux and ZFS on FreeNAS that I need to be concerned about? (that come
into play when pushing/pulling datasets)

~~~
e12e
> Can I use something like rsync.net to > push my datasets up as a backup

Yes, but depending on the size/value of the dataset, rsync might not be the
best option.

Even after recent price reduction (Yay!), and with a hn discount (Yay!) - I
still consider rsync.net as a _personal_ backup service a little
overkill/expensive past 100 GB or so. And likely you could just handle
anything on the order of 1 to 2 tb locally anyway (spare disks).

You technically could use it for 10+tb of data... But uploading and
downloading is going to be slow unless you have a real uplink (ie at least
1gbps).

And if you did, unless you need filesystem level Metadata - rsync might be a
better option than zfs send/receive (even though zfs is cool).

>, rebuild my server, and then pull them back down into a reorganised/renamed
tree of datasets?

Again, if you just need the data back - you might be well served with rsync or
restic (both choices should be orthogonal to using rsync).

> Are there compatibility issues between ZFS on Linux and ZFS on FreeNAS that
> I need to be concerned about?

The world is divided in Oracle and not-Oracle - AFAIK there shouldn't be major
differences between those two - that I believe now both follow upstream
zfsonlinux.

However - I believe rsync.net still doesn't support zfsonlinux/openzfs
encryption (zfsonlinux 0.8+). While locally it's possible to encrypt the disk
"under" zfs... Native encryption is tempting.

And you would of course never send anyone data without encrypting it?

~~~
rdc12
The switch of FreeBSD's ZFS master being ZoL is still work in progress, and is
not the default ZFS in FreeBSD yet. (FreeNAS being FreeBSD + Configuration +
Management Software).

A fair number of the ZoL 0.8 features (encryption, allocation classes being
two prominent examples) are not available currently for FreeBSD

~~~
agapon
Allocation classes are available in FreeBSD CURRENT snapshots.

------
mckmk
Sanoid seems like a great tool. It was at the top of my list when I was
looking for this kind of solution.

I ended up going with zfs_autobackup -
[https://github.com/psy0rz/zfs_autobackup](https://github.com/psy0rz/zfs_autobackup)
because it doesn't need anything set up on the computer to backup. Just a
script on the backup server that does pull backups.

I'm not trying to hijack the thread but I've often appreciated links to
similar software in threads like this.

~~~
Youden
The issue I have with that is that if everything is done by the backup server,
it has the ability to both create and destroy (prune) datasets (snapshots),
which means it has the ability to erase all of the data and all of the
backups.

One thing I like about Sanoid/Syncoid is that if you set up Sanoid on both
sides and use Syncoid to pull, the backup server only requires the ability to
send snapshots (which you can configure with "zfs allow"). The backup server
also doesn't require root in this case. This means neither server can delete
the other's data.

And you still get all the automatic snapshot creation (done on the primary
server) and pruning (done on both servers).

------
hartzell
For FreeBSD users, it's available from ports:

[https://www.freshports.org/sysutils/sanoid/](https://www.freshports.org/sysutils/sanoid/)

(I'm the port maintainer, open to improvements)

------
beagle3
Only tangentially related, but seems like a good thread to ask:

Does ZFS (or any other FOSS file system) have a setup like Isilon or NetApp,
where you could have any 2 disks in a box of 8 or any box out of say 6 die,
and still not lose anything other than bandwidth?

~~~
antongribok
I would say ZFS would be a lot closer to NetApp.

For Isilon, a better comparison would be Ceph. With Ceph you can have a
cluster of 10s or 100s of servers, and you can lose 1 disk, or 1 server, or 1
rack or servers and it will recover very quickly.

As long as your minimum requirements for your type of replication or erasure
coding profile are satisfied, your data stays safe and available.

------
linsomniac
This looks pretty solid. On one of my machines I'm using "zfs-auto-snapshot",
which works great for something I can just install and not have to manage, for
getting some history. For replication of snapshots, I haven't done that
recently, but in the past I just used some scripts I had written to do one-
time migrations. That was a decade ago. This looks much more capable than
those scripts.

~~~
kchr
Yeah, there are base templates for configuration and they can be overridden on
a single (or recursively on all children of a) dataset where needed.

------
e12e
Oh my. It's perl. Might be worth noting.

For those that actually use perl - I know perl has had (fairly sane?) package
management for a while... Is it reasonable to use system perl version in
anger?

I would typically reach for a version manager/virtual env with python or ruby
- do you, with perl as well?

I'm fairly happy with asdf day to day.

[https://asdf-vm.com/](https://asdf-vm.com/)

------
kchr
Great piece of software; the templating model for configuration is neat and
works out well for my case.

------
paxswill
I love using Sanoid+Syncoid for keeping my laptop (with Xubuntu on ZFS) backed
up to my NAS (running FreeNAS). In addition to the periodic snapshots Sanoid
manages, I also have a small script (shameless plug [0]) that hooks into APT
to take snapshots right before upgrades occur. It's already saved me decent
chunk of time that would've been spent rebuilding from a botched upgrade.

0: [https://github.com/paxswill/zfs-apt-
snapshot](https://github.com/paxswill/zfs-apt-snapshot)

------
parshimers
I wanted to use this tool for backing up my laptop to my NAS, however it only
allows SSH to be used for data transfer, which is a real bottleneck for speed.
It basically means you can only transfer as fast as one core can decrypt or
encrypt data on either end of the pipe.

I ended up using
[https://github.com/oetiker/znapzend](https://github.com/oetiker/znapzend)
instead, because it can use mbuffer for the actual data transfer, which is
much faster.

~~~
kogir
Is SSH speed a real limitation you’ve measured? I was able to easily saturate
a 1 Gbps link for days using ZFS send over SSH with an Intel Atom when
initializing my remote backups.

~~~
parshimers
Yes. I have an Atom on one end of the pipe as well, but it's over 10Gbe and to
a 6 drive mirrored and striped vdev. Using SSH as the transport it would only
approach 100MB/s, using mbuffer it can max out the storage at about
250-275MB/s.

------
r1ch
I've used a few different ZFS snapshot management tools, the one feature I'm
missing is something to assist with data retrieval. I'd like to be able to
specify a dataset, path and date/time and the tool would mount the closest
snapshot to the specified time and pull the appropriate data for me.

~~~
rainbowzootsuit
Not exactly what you're asking I think, but in Solaris-derived OSs the
snapshots are exposed by default as previous versions in windows.

You can also use SAMBA to expose the snapshots as previous versions in
windows.

SAMBA general documentation: [https://www.samba.org/samba/docs/current/man-
html/vfs_shadow...](https://www.samba.org/samba/docs/current/man-
html/vfs_shadow_copy2.8.html)

Randomly found ZFS specific: [https://blog.chaospixel.com/linux/2017/09/zfs-
auto-snapshots...](https://blog.chaospixel.com/linux/2017/09/zfs-auto-
snapshots-and-windows-shadow-copies-with-samba.html)

------
timeattack
Another one, which creates backups on locally connected backup pool —
[https://github.com/reconquest/zeus](https://github.com/reconquest/zeus)

------
polskibus
Is this a good fit for an on-prem small data center backup solution?

------
grafelic
I've been using it for my daily zfs snapshotting needs for a year. Rock solid
software. Thanks.

------
crashbunny
sanoid is great. I asked how to stop sanoid from snapshotting docker created
datasets in rpool/var/lib/docker when rpool has the recursive flag, and they
added a feature to do it the next day.

------
kuon
What advantages does it have over zrep?

------
techslave
yawn. hasn’t this been done a million times over?

~~~
dang
Please don't be a jerk on HN. Also, note this guideline:

" _Please don 't post shallow dismissals, especially of other people's work. A
good critical comment teaches us something._"

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

~~~
techslave
in my defense, i wasn’t dismissing his work. i was wondering why it merits HN
attention. it’s completely ordinary stuff. the world needs ordinary stuff but
this isn’t buzznews “the 3 surprising tricks about zfs snapshots”.

