Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Btrsync – btrfs snapshot replication made easy (github.com/andreittr)
2 points by andrei-ttr on Jan 7, 2023 | hide | past | favorite
tl;dr: btrsync is rsync but for btrfs snapshots; you type in source & destination, it figures out everything else; check it out at https://github.com/andreittr/btrsync

# Background

I've been using btrfs for nearly a decade now, setting up daily or even hourly automatic snapshots of various parts of my system. Being able to go back in time is great for when I mess something up and forget how I got there. That happens. Snapshots also happen to be a good starting point for backups.

Snapshots aren't really backups until you transfer them off-device and put that other device somewhere else (ideally offline and off-site). Luckily, btrfs has incremental `send` and `receive` to handle this. You just tell `send` what volume to use as a base and you have incremental transfers, magic!. Assuming the same base volume is there on the `receive` end. And I mean the exact same. Fair enough, so I got btrfs on external storage and proceeded with backing up.

At first, shell one-liners and basic scripts worked out well...-ish. String processing in shell scripts is cursed, so I pulled out Python, leading to the first incarnation of `btrsync.py`.

But edge cases kept popping up, and constantly babysitting the process got tedious fast, there had to be a better way. An automatic way. Looking around, there were some btrfs automation tools out there, even GUIs, but they seemed to focus more on rollback than backup. I also didn't want to overhaul my existing snapshotting setup. What I wanted was something simple that I could invoke on a whim and it would figure out everything, something like rsync, but for btrfs. I was surprised it wasn't a thing yet.

So, in the spirit of scratching my own itch, I got to work on btrsync v2, a complete rewrite needing a bit less handholding and with shiny new ssh support! Because I couldn't be bothered to connect my backup drive to another computer. I'm not entirely sure what was going through my head at the time, if anything, but the code was a mess; it would fail mysteriously, hang forever if stuff wouldn't finish in the right order, and leave zombie processes around if interrupted at the wrong time. A mess. But it also kinda worked, heroically handling hundreds of gigabytes of backups with only the occasional diving-down-a-rabbit-hole bug hunt when it misbehaved. It's OK, it's good, it works, leave it be, I said.

But it was not OK, it wasn't even close to reliable, and any attempt to fix the code now, years later, feels like archaeology, so it was time to do it properly. Well, third time's the charm, because btrsync round 3 went surprisingly good, hence this post. Subvolume discovery, diff-ing, parentage, etc. are all handled automatically now, no more handholding, and finally a CLI as simple as rsync or scp.

I reckon I can't be the only one needing something like this, so after some quick code cleanup, doc writing, and various other admin stuff, I've put btrsync out in the wild. Ready to lend a hand to anyone who, like my past self, just wants something simple and straight-forward to copy their snapshots around.

Give it a spin and let me know what you think!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: