

Ask HN: Using S3 for personal data backup? - ryanteo

Hi HN, 
I would like to ask whether anyone has experience using S3 for personal data backup (documents, photos, video) and work.<p>Background: Just experienced an external hard drive failing on me. Traumatic and irritating =)<p>Hope that you can share how you've set it up (rsync?), some examples of the data you've stored and the costs that you are paying each month.<p>Luckily for me, Amazon just opened a data centre in Singapore =p<p>Thank you!
Ryan
======
cperciva
I'm using S3 for personal data backup.

How I set it up: <http://www.tarsnap.com/>

~~~
g0atbutt
Doesn't the person that created tarsnap post here?

~~~
mahmud
I hope that was nod to the putnam story, otherwise WOW, Colin must get 'em all
the time.

~~~
cperciva
I don't get them all the time, and I doubt it was that. The account is new
enough that he probably hasn't seen the Putnam story.

------
metalwheaties
Do you know about <http://www.wuala.org> ? It's free if you share your disk
space, it's redundant, and it's fast and easy. Backups, rsync, and regular
file storage via a browser in a Java based client (runs everywhere I have
tried) make this universally good. I never paid a cent to them, shared a
couple of machines worth of 50GB of extra space that is up 100% of the time,
and I have > 100GB of space available to me.

Linux backups are really easy: Wuala's client uses fuse to mount itself as a
file system, so you can simply do periodic tar, or you can let their own
backup mechanism back up your stuff to a special folder they maintain
perfectly. You can get to your backed up files anywhere in the world via
Wuala.com. This feature has saved me gobs of effort a bunch of times already.

~~~
metalwheaties
BTW, I have no affiliation with Wuala. I just like it.

~~~
ryanteo
Thanks!=) I'll look in that... Did not know there were so many options
available.

------
hga
I rather like <http://rsync.net/>. My critical, need to keep it offsite data
backup requirements fit it well, there's one simple monthly charge and I can
use rsync plus a handful of blessed shell commands to manage it. They also
figured out a way to support git, which I use for projects that are too
incomplete or unpolished to put up on GitHub. I looked very hard at S3 but
it's rather inflexible for exactly what I want to do.

That said, I'm only storing my parent's photos and videos of their
grandchildren (I'll lose all my media if my apartment is _truly_ wiped out by
a tornado, then again I'm unlikely to survive that). If you need to backup a
_lot_ of stuff S3 will be quite a bit cheaper (for me either is pocket change
as of now).

------
vitovito
bradfitz's brackup supports securely backing up data to S3 (and elsewhere),
including incremental backups.

<http://code.google.com/p/brackup/>

I don't actually use this, I just followed its development. For my backups, I
have a script that spins up an Amazon EC2 instance, mounts an EBS disk and
does an old-fashioned rsync of my entire machine.

This does mean I pay for the entire disk all the time every month, plus the
EC2 time, and it's probably slower and more expensive than an equivalent
brackup, but I also didn't have to think about it.

Right now, the EBS disk isn't bootable, but I could make it so and in theory
just boot up an EC2 instance and immediately have my entire machine at the
state it was last backed-up as.

------
briandoll
I used to use S3 to backup data on my mac mini via JungleDisk. I was paying
somewhere around $10-15/mo in storage costs after the initial fee of uploading
the bulk of the data. Unfortunately after a system upgrade (not sure when or
what specifically), the automatic sync w/ JungleDisk just stopped working. For
several months none of my data was being copied back to S3.

What I really need is simplicity that _just works_. I found that in Dropbox
and haven't gone back since. I use it to back up ~100Gb of data. On my other
machines I just choose not to sync the larger directories and everything works
perfectly.

~~~
BerislavLopac
I absolutely, positively second DropBox. It integrates seamlessly, and you may
have 100GB for only $200 a year.

But th best thing is that you can try it for free: it gives you 2GB initially,
and an additional 250MB (up to 10GB) for each new user you refer. (E.g. if you
apply via this link, I'll get an additional 250MB:
<http://www.dropbox.com/referrals/NTIxMzc2ODk> _hint hint ;)_ )

~~~
ryanteo
Thanks! I've been introducing dropbox to friends too (not just because of the
referral=) ) It just works. However, I was wondering whether it's cheaper to
use S3 instead, since I intend to store around 200 GB worth of files.

------
foundbobby
I just set up s3fs on my linux server
(<http://code.google.com/p/s3fs/wiki/FuseOverAmazon>) and pointed it at Google
Storage for Developers ([http://code.google.com/apis/storage/docs/getting-
started.htm...](http://code.google.com/apis/storage/docs/getting-
started.html)). It's intended for use with S3 as the name implies of course.
It's been working flawlessly so far :)!

------
dotBen
CyberDuck (free, for Mac) otherwise JungleDisk ($$) to connect your machine to
S3.

Don't backup your data to Singapore even if you are local, it will cost you
more (compare costs across S3 DC's) and you don't need backup data to be close
to you in the network.

S3 can work out expensive if you want to host a lot of data. Sometimes it can
be cheaper to buy one of those 'unlimited' backup accounts with Mozy/etc as
after a while the cost will be cheaper than Amazon.

~~~
amock
The EU west region is just as cheap as the US standard, so if you want to
backup off continent and not pay more you can. As far as the unlimited backup
plans go, how does their durability compare to S3 or SpiderOak? It seems like
if they're so much cheaper they can't have as much redundancy.

~~~
dotBen
_It seems like if they're so much cheaper they can't have as much redundancy._

They probably don't have the same amount of redundancy as S3 but then
redundancy is a subjective thing.

A backup service might claim that your data is only stored in one place (ie no
redundancy) but that it's very stable.

Amazon S3 might be more diverse but it's also volatile - Amazon claim that
they cannot guarantee not to loose data.

~~~
lsc
nobody can claim that they 100% won't loose your data. (I mean, they can claim
that, but everyone has problems from time to time. Bad things happen. Magnetic
storage is fragile.) Amazon, from what I understand, has one of the best
documented systems; and with six (6!) copies of your data, well, that's better
than any other storage system I know of, and their price is pretty good.

Now, amazon is pretty expensive if you access the data often... but beating
their prices for archive storage (with six copies!) is damn hard. I've been
looking at competing with them for a while, and it looks like what I might end
up doing is selling space with a significantly lower level of redundancy (2-3
copies) at a lower rate, with a significantly lower network transfer charge.

Even so, the amazon system is really nice.

------
rphlx
Consider using duplicity to backup to gmail IMAPS. 7GB in a luxury, sea-side
GOOG datacenter, free of charge forever. Need >7GB? Create another gmail acct.

The problem with paid services is that if you have even a temporary credit
card/billing problem, they may nuke your data. Also, they're probably less
reliable than gmail.

------
cpswan
I've been using www.SMEstorage.com for this. They support multiple storage
clouds as back ends, and have DropBox style tools that give me automated sync
between multiple machines (and mobiles). Keeping all of my 'work' data safe
and in sync between my desktop and laptop is costing me cents a month.

------
CatalystFactory
My friend runs: <http://www.syncplicity.com/> \- might be of use.

------
irv
another vote for tarsnap - can't complain really. i knocked up a simple
Haskell app to manage the process (I'm learning Haskell, so comments and
suggestions welcome) if you're interested @ <http://github.com/irv/tarsnap-
backup>

------
hellweaver666
Isn't that basically just what Dropbox does?

------
epynonymous
i use <http://www.mozy.com>

