
Causing ZFS corruption for fun, profit, and quality assurance - binwiederhier
https://datto.engineering/post/causing-zfs-corruption
======
pizza234
ZFS volumes can also be backed by files. In this case, the volumes can be
corrupted by opening the backing file in a text editor, and perform a
search/replace. This is simpler but of course less interesting :-)

Another interesting article I've mentioned in another post, is a demonstration
of ZFS' self-healing capabilities:
[https://blogs.oracle.com/timc/demonstrating-zfs-self-
healing](https://blogs.oracle.com/timc/demonstrating-zfs-self-healing).

~~~
hnlmorg
Being able to back volumes by files is an amazing feature for testing. I've
lost count of the number of times I've done this to prove some documentation
works before applying it to the actual storage array in question.

------
the8472
Some device-mapper virtual block device types also provide ways to simulate
error conditions. For bad block errors there is dm-dust.

[https://www.kernel.org/doc/Documentation/device-
mapper/](https://www.kernel.org/doc/Documentation/device-mapper/)

------
derekmhewitt
This was an interesting read! I hope they do a follow up article on how they
automatically manage the rest of their recovery process.

~~~
pizza234
If with "recovery", you intend recovering that file, it's not possible, since
all the copies available in the mirror are corrupted.

In this case, the recovery (repair) of the system (that is, to bring it to a
clean state) simply consists in deleting the file (see
[https://docs.oracle.com/cd/E26505_01/html/E37384/gbbbc.html#...](https://docs.oracle.com/cd/E26505_01/html/E37384/gbbbc.html#gbctx)).

If there was an intact copy of the file available in the mirror, then one
doesn't need to do anything, as ZFS is self healing:
[https://blogs.oracle.com/timc/demonstrating-zfs-self-
healing](https://blogs.oracle.com/timc/demonstrating-zfs-self-healing).

~~~
kchamplewski
Presumably they're referring to Datto's recovery process (i.e. recovering
automatically from secondary or tertiary backups), as opposed to just the ZFS
mirror.

