
A backup rotation filter for the Unix shell - closeneough
https://git.sr.ht/~apreiml/prunef/
======
adrianmonk
> _To list backups that will should be kept use the --invert option._

May I suggest changing the name to something more direct? Calling it
"\--invert" means the user must think through what the default sense of the
test is, then negate that in their mind. Not exactly a tough mental task, but
people do make careless errors.

Perhaps something like "\--list=keep" and "\--list=discard" (with the default
being "discard").

Also, typos:

"will make prunef to keep" \--> "will make prunef keep"

"list backups that will should be" \--> "list backups that should be"

~~~
closeneough
Thanks for the input and for pointing out the typos. They are fixed now.

I will consider changing the invert flag, but I'm not that happy with
something like "\--list=...". There will be only two modes with discard being
the default one. So imho there should be only one flag to switch to the non-
default mode.

~~~
closeneough
I've decided for --list-kept. --invert will be supported to not break scripts.

------
yjftsjthsd-h
Handy looking tool:)

Meta: I'm excited to see sr.ht starting to pop up in the wild like this:) I
hope this is part of it starting to take off.

~~~
brobot182
The layout, at least on mobile, could use some work

~~~
leni536
What problems do you find? I can't find any.

~~~
OJFord
Not GP, but it gives me about 5-10% horizontal scroll, for the benefit of the
last few letters of 'contributors' and background to 'summary'.

It's fine though, I leave most blame at iOS' door. Everything looks too big on
this temporary iPhone SE, and I'm not allowed to zoom out, default to Firefox,
or use any extension or 'content blocker' in it even though it's forced to use
Safari to render. (/Rant..)

~~~
saagarjha
> It's fine though, I leave most blame at iOS' door.

I think this is just some missing CSS to hand this case.

> Everything looks too big on this temporary iPhone SE, and I'm not allowed to
> zoom out, default to Firefox, or use any extension or 'content blocker' in
> it even though it's forced to use Safari to render.

On my very much not temporary iPhone SE, I can use content blockers and zoom
out…

~~~
OJFord
Not in FireFox you can't.

And I mean 'zoom' out from the default, e.g. on desktop I browse most sites at
80% in FF, some 67 or 50, fewer at 100.

On my in-for-repair Android phone, the smallest system UI/font setting is
smaller, and FF is allowed extensions and to use its own renderer, so I have
control over that.

Everything just seems like I'm using a largified accessibility mode. And
typing - impossible to place the cursor mid-word? So if suggested corrections
are wrong, no choice but to delete and re-type the whole thing. And no select
all? So if I decide not to post such a rant, I have to fumble with the two
cursors and move one to each end myself.

------
speedgoose
I use rotate-backups with temporary files and your script is an interesting
alternative.

[https://pypi.org/project/rotate-backups/](https://pypi.org/project/rotate-
backups/)

------
inshadows
FYI the interface format seems to be inspired by borg-prune[1].

[1]
[https://borgbackup.readthedocs.io/en/stable/usage/prune.html](https://borgbackup.readthedocs.io/en/stable/usage/prune.html)

~~~
closeneough
Yes, this was a starting point for me. I was referencing to borg prune in my
initial readme, but I dropped it because the algorithm works differently and I
wanted to avoid confusion.

------
usr1106
The original title reads "... for your Unix shell", suggesting that the code
is portable to many shells. I have not verified that claim.

"... for the Unix shell" makes little sense. When I used Unix my shell was
csh, later tcsh. Nowadays my shell is bash in most cases and dash in some more
limited environments. Either case, "the Unix shell" does not exist.

------
epx
Did something similar but employed a Fibonacci sequence, using the hour as the
unit (but could be a minute, or a second), to groom a collection of snapshots
maintaining a sensible timeline:
[https://epxx.co/logbook/entries/fibo_en.html](https://epxx.co/logbook/entries/fibo_en.html)

------
bArray
If anybody else is using a similar method, I think it's also good to encrypt
the backups:

    
    
        tar -zcv <SRC_BACKUP> | gpg -c --batch --passphrase <PASSWORD> -o <DEST_BACKUP>.gz.gpg
    

In my experience the encryption part doesn't add any extra time on a modern
machine, with the spinning disk being the slowest cog.

~~~
NieDzejkob
There's a lot of complexity in gpg that's unnecessary for this usecase, I'd
use age instead.

[https://age-encryption.org/](https://age-encryption.org/)

~~~
voidmain
gpg is widely agreed to be a train wreck, but age is not generally suitable
for encrypted backups because it doesn't offer any form of authentication (so
an attacker can replace a backup with a malicious one).

~~~
cyphar
Can you elaborate on the threat model a little bit? I'm struggling to
understand how you can protect against this in a way where an attacker that
both knows the relevant encryption key (whether public or secret, depending on
whether the crypto is asymmetric) _and_ has write access to the backup
location.

If you sign the backups with some distinct key of the backup server, why
wouldn't the attacker have access to those keys too (in the above scenario
they already have access to the keys that the backup server is using for
encryption).

I know there's an open issue in age for adding authentication[1], so there
clearly is some threat this would protect against but I can't figure it out.

[1]:
[https://github.com/FiloSottile/age/issues/59](https://github.com/FiloSottile/age/issues/59)

------
djsumdog
This is awesome. I currently use duplicity for backups to BackBlaze, but my DB
backups I just place a files and I've just been pruning old one's manually for
a while. Something like this, that lets me specify the file format, would be
perfect!

~~~
stevekemp
For database backups I've always done the simplest thing, I take a daily dump
of "unchanging" databases:

* /var/backup/db/db1/monday.sql

* /var/backup/db/db1/tuesday.sql

* /var/backup/db/db1/monday.sql

For databases that change more frequently I instead backup every 1, 3, 4 hours
as appropriate:

* /var/backup/db/db2/monday/00.sql

* /var/backup/db/db2/monday/04.sql

The appeal of this is that I always have "local" backups, and I don't need to
consider rotation at all each one gets the most recent copy when it runs, and
I have an alert/alarm to make sure files are recent enough that things aren't
broken. I appreciate that if your databases dumps are 600Gb each, or something
similarly sized you'd waste a lot of space, but for small things the
simplicity of this approach is a good win.

(These get copied offsite as part of the backup of the whole filesystem. In
the past I used to backup only some stuff, that failed the first time I tried
to restore a mailserver and didn't have /var/lib/mailman archived! These days
I explicitly backup "/" excluding only /tmp, /proc, /sys, and /dev.)

------
ggm
Could you leverage information in zfs snapshots metadata to drive things like
this? The copy on write semantics inherent seem plausibly related.

(Zfs snaps make awesome backups too, but unlike tar are inherently tied to
zfs)

------
FrancoisBosun
Similar concept, in Ruby, acting as a filter in a pipeline, by me:

[https://github.com/francois/surrender](https://github.com/francois/surrender)

------
anonsivalley652
PSA: Be sure to test every backup completely before succeeding a backup job,
or it's not a backup.

------
aquabeagle
tarsnap desperately needs something like this included with it.

~~~
aorth
Also see tarsnapper. I've been using it for years.

[https://github.com/miracle2k/tarsnapper](https://github.com/miracle2k/tarsnapper)

