

Bup – An efficient backup system based on the git packfile format - tambourine_man
https://bup.github.io/

======
zeograd
When I had to pick a backup system, I considered Bup until I saw that there
were no way to prune old backup (
[https://github.com/bup/bup/blob/master/README.md#things-
that...](https://github.com/bup/bup/blob/master/README.md#things-that-are-
stupid-for-now-but-which-well-fix-later) ).

This is really a stopblocker for me.

Obnam ( [http://obnam.org](http://obnam.org) ) is a similar tool but support
forgetting old generations. However, it still suffers for youth problems and
tends to corrupt backup repository when pruning old data on a remote server.

I'm really looking forward for a mature and feature full backup system based
on git principes. Bup or obnam might be one of those.

~~~
rlpb
I wrote ddar before I knew about bup. It works in a similar way, but uses
sqlite and flat files for chunk storage, so removing old archives isn't a
problem. I'm not aware of any corruption issues in ddar; I rely on sqlite for
integrity.

I modeled ddar after Tarsnap.

~~~
tambourine_man
Don't know if you are aware, but
[http://www.synctus.com/ddar/](http://www.synctus.com/ddar/) is currently down

~~~
rlpb
I'm aware, thanks.
[https://github.com/basak/ddar/wiki](https://github.com/basak/ddar/wiki)
contains the content, thanks to a kind contributor. I should probably
deprecate the old URL and remove references to it.

~~~
zachlatta
Perhaps you could have it redirect to the wiki?

------
mynegation
I do a monthly offsite backup. I use rdiff-backup for plain text and 7z with
encryption for data I want to keep private, but I am on the lookout for more
efficient solutions. I looked at bup, but two things stop me from using it: no
encryption, and inability to delete old backups to reclaim space. To me, attic
([https://attic-backup.org/](https://attic-backup.org/)) looks attractive so
this is what I am going to try soon.

~~~
wampus
I've used rdiff-backup daily for years, but recently needed to store backups
on machines I don't control, so I gave duplicity[1] a try. It encrypts,
doesn't need to be installed on the target host and is simple to use if you're
already familiar with rdiff-backup and gpg. Test restores were simple, but
were all first generation, so not really a good test (but there are hints
you'll want to prune backups to one month's worth). I haven't figured out an
acceptable way to automate it, so I've been running it manually on an
infrequent basis while I evaluate it. Give it a look, if you haven't already.

~~~
aroch
One of the downsides with duplicity is you have to provide your GPG privkey
password in ENV in order for dedupe with GPG encryption to work.

~~~
sciencerobot
You don't. You can use gpg-agent with --use-agent.

    
    
      duplicity inc \
        --sign-key $PGP_PRIV_ID \
        --encrypt_key $PGP_PUB_ID \
        --use-agent \
        ${source} ${dest}

~~~
aroch
Which would require interactive usage or for you to save your passphrase in a
system keychain, no?

------
gh02t
Since I see various different backup utilities on here from time to time, I'm
curious if anybody uses Bacula (or Bareos)?

It does encryption, deduplication, scheduling, pruning etc. I use it at home
and run the director and storage daemons in a jail on my FreeBSD NAS, which
coordinates all of my computers together for backup regularly. I then back up
the encrypted backups to Crashplan.

I tried other stuff (especially duplicity), but they just didn't work very
well for me. Restoring encrypted backups with duplicity was especially slow
and annoying and even using the duply frontend it was rather fickle.

Bacula has some disadvantages though... for one it is a bit complicated to get
working. It's partitioned into multiple daemons, each of which handles a
different task and which can be on different machines. This makes it flexible,
but also makes the configuration more complicated. It also works best if you
have a central backup server that can run the director and storage daemons
24/7\. You can set it up to work all on one computer and just dump to a "dumb"
storage drive, but it is really designed to work with a central server.

~~~
sliken
Last I checked bacula doesn't do deduplication in any normal sense of the
word. It allows you to run a base job and then back up diffs, that's a far cry
from normal dedupe.

~~~
gh02t
Oh, yeah you're right. I don't use that feature, so I didn't really know the
limitations. But looking at it, base jobs, while useful, are definitely not
true deduplication.

------
victorhooi
Currently, we're using Duplicity for our backups, going to Amazon S3.

I've looked a some of the newer options like Bup, Attic, Obnam etc.

However, none of them seem to support S3 as an endpoint, which is a shame.
They all seem to need a full-fledged *nix server on the other end.

Does anybody know of any of these new-fangled de-dupe backup apps that works
with just S3?

~~~
rrrrtttt
S3QL may be a good fit for your needs. It's a FUSE filesystem for S3 that
supports encrypted dedup snapshots. You can do backups by rsyncing your
current state into the filesystem and snapshotting it. I considered using it
for my backups but ended up choosing Attic, so I can't say how well it works
in practice.

~~~
Freaky
Unfortunately S3QL uses MAC-then-encrypt[1], which is pretty strongly
discouraged[2]. Very nice there's a detailed writeup on the details in the
docs though, wish more projects did that.

1: [http://www.rath.org/s3ql-docs/impl_details.html](http://www.rath.org/s3ql-
docs/impl_details.html) (last paragraph) 2:
[http://www.daemonology.net/blog/2009-06-24-encrypt-then-
mac....](http://www.daemonology.net/blog/2009-06-24-encrypt-then-mac.html)

------
dochtman
Previously:

[https://news.ycombinator.com/item?id=7245297](https://news.ycombinator.com/item?id=7245297)
(a year ago)

[https://news.ycombinator.com/item?id=8620236](https://news.ycombinator.com/item?id=8620236)
(3 months ago)

------
StavrosK
Here's a writeup of my quest for the perfect backup tool:

[http://www.stavros.io/posts/holy-grail-
backups/](http://www.stavros.io/posts/holy-grail-backups/)

Summary: Use attic.

------
yiyus
I wonder how this compares to venti. Being able to mount with FUSE can be an
interesting feature, specially if it can work well even over the network
(maybe with some kind of cache, like fossil).

~~~
prouleau
bup can mount the backups with FUSE too. It also have a web interface with
WEBDAV support, which our graphists uses to recover their old creations.

------
lucaspiller
Kind of related: What do you use to backup binary files like photos? At the
moment I've just got copies on old USB drives, but I'd like to put something
on AWS Glacier too.

~~~
rlpb
git-annex. It supports multiple external remotes, including USB drives,
Glacier[1], bup and ddar[2]. You can keep files on any combination of the
external remotes for redundancy and cost management. It keeps track of what is
where, so you don't have to.

The only catch is that you have to make sure to back up the (metadata-only)
git repository itself also, and maintaining this backup on anything that is
not a direct filesystem is painful.

[1] Through my tool, glacier-cli, for which git-annex has native support.

[2] I wrote ddar and the git-annex support for it.

~~~
rcthompson
So git-annex now has support for efficiently deduplicating backends? Might be
time for me to give it another look.

~~~
rlpb
For "local" storage as supported by bup and ddar, yes. Generally for external
remotes: not yet, AFAIK. [http://git-
annex.branchable.com/design/assistant/deltas/](http://git-
annex.branchable.com/design/assistant/deltas/) has some design thoughts, so I
think it is planned.

------
detaro
slight OT, but is there anything that adds transparent encryption to these
backup solutions? If you're willing to live without in-file diffs, a FUSE
filesystem that presents all files as GPG-encoded or something similar would
be interesting, but I've never seen anything convincing like it.

------
meapix
I've being using this for a while now. I even backup movies, it's saves on
space.

