

Arq (Mac backup) now works with Glacier - sreitshamer
http://www.haystacksoftware.com/blog/2012/11/arq-cloud-backup-for-mac-adds-support-for-amazon-glacier/

======
cmer
This looks great! Would it be possible to add the total estimated cost for
retrieval? At first sight it looks like downloading at a slower rate is
substantially cheaper although it is not. I think this would give a better
idea to your user how much downloading the data will end up costing.

~~~
sreitshamer
Yes, that's a good idea, probably for both S3 and Glacier retrieval.

------
c0nsumer
This looks really interesting to me. One thing that I'm not quite clear on is
whether or not backups made using Arq to Glacier can be restored using other
software.

I'm mildly concerned about what could happen if I need to restore software in
three years on a new version of OS X (or a different OS entirely), possibly
after sreitshamer has been hit by a bus and not updated things.

~~~
leejoramo
Arq uses its own format. However, the developer of Arq has provided an open
source restore tool on github:

<http://sreitshamer.github.com/arq_restore/>

[http://www.haystacksoftware.com/blog/2010/04/open-source-
arq...](http://www.haystacksoftware.com/blog/2010/04/open-source-arq-restore-
utility/)

Edit: the files on Github have not been updated in 3 months. So this most
likely can't restore Glacier backups.

~~~
sreitshamer
I'm working on an update! Sorry it's not done yet.

------
furyg3
So how does this work in practice? Can I backup directly to Glacier? Or do I
have to backup via S3 first?

I use TimeMachine, but would like to have a "my house burned down" option that
lives online somewhere...

------
thebigrace
Great work from the founder. Arq is some really great software, highly
recommended from experience - now to see what it's like on Glacier :/

------
magic5227
Seems like bandwidth costs would kill this, restoring a TB is $120 in
bandwidth alone, then you have the retrieval fee.

I agree with some others that the cheapest for now is buying more HD and
keeping them in sync with crashplan or something else, until Google Drive
drops a bit further in price for 1tb.

~~~
cschmidt
I do a rotation of hard drives in my house. I think it would be useful for the
"my house burns down" scenario. In other words, I will probably never need the
restore, but if I do, cost really isn't an issue.

------
chucknelson
I wasn't very interested in Glacier at first, but the more I hear about it,
the more I'm convinced it's a perfect companion to time machine backups. Why
pay $13/mo for CrashPlan or some other similar service when I can pay a
fraction of that for Glacier?

Of course, this makes sense for me because my backup is rather tiny (~300 GB).
The "unlimited" plans at backup services are worth it in certain scenarios I
guess.

~~~
prof_hobart
Where are you getting $13/month for Crashplan? Their site has it at $3/month
for unlimited, or $6/month for up to 10 computers.

~~~
subpixel
Crashplan is cheap b/c they make it quite difficult for the average user to
_actually_ backup all of their data. The interface is all sorts of confusing
to non-experts, and I was embarrassed to realize I had fallen for a UI trick
that made me think my drive was backing up, when in fact it was only my user
directory. I'd made the same mistake on the 6 computers in the family, and had
to do a bunch of phone-tech support with in-laws.

I paid for a year, so I'm using it now as additional backup, with my primary
backup on a local HD and Arq.

------
blackdanube
For people who just want to put their data on Glacier without having to worry
about the details, there's also Bit Chest at <http://bit-chest.com/>

It works more or less the same as Glacier, only with a lot of cushion around
it.

\--- edit: Disclaimer: I am the owner of Bit Chest.

~~~
PanMan
And it is 5-10x as expensive as Glacier itself... Also, assuming Arc stores
your data in a 'standard' way, I only have to worry about Amazon going out of
service. Here there is also the risk of this small german company.

~~~
blackdanube
That is indeed true, there are currently no mechanisms in place for ensuring
users' data in case of insolvency. However, we're working on getting an escrow
agreement in place that would ensure that Glacier vaults can be transferred to
their "owners" in the case of insolvency. Additionally, there are some
provisions in german GmbH laws that give warning before insolvency occurs and
allow clean processing in the case.

Concerning the price, it is indeed a multiple of pure Glacier. However, they
include, apart from lots of convenience, also all auxiliary costs that you'd
have to manage yourself for a "pure" Glacier account. As noted in another
comment, you'd have to keep an eye on retrieval cost, early deletion and
additional S3 storage for metadata.

If you're not interested in tinkering with IAM keys and retrieval rates, I
still think Bit Chest is a good value proposition.

Disclaimer: I am the owner of Bit Chest.

~~~
davidcollantes
You should have pointed the disclaimer on your original, blatantly
advertising, comment.

I prefer to use the space I have on Amazon S3, under my own account with them,
which I pay directly to them, than using some other obscure, third party,
close beta service.

What is "ct" on your pricing? Just curious.

~~~
blackdanube
I edited to add the disclaimer. Thanks for calling me out on that.

I understand that most people on HN will probably use Glacier directly, and
that is totally OK. The service that I am offering is not "making it
possible", rather it's "making it simple, so much so that your grandmother can
use it". You do use Dropbox in the same way, don't you? Also, the reason it's
obscure is that is less than a week old and has less than 40 users. I wouldn't
expect any offering of that age/size to be well-known, so I'm quite satisfied
with my obscurity.

ct is cents. For Europe, that will be Euro cents, all other areas will be
billed in USD cents.

------
jcapis
I'm disappointed they chose to charge an upgrade fee to former customers.
Especially after a year of serious bugs and many users having to replace their
backups completely. I'm not sure if I'd still recommend it, but I am happy to
see the Glacier support - it looks like a natural fit.

------
Osmium
Given the retrieval costs, it seems to be that Glacier is better for an "if
all else fails" backup. A cheap-to-store, last resort when your other backups
fail. Still, it's nice that Arq calculates the retrieval fee for you.

~~~
sreitshamer
(I'm the guy behind Arq) Yes. Unless you want to retrieve very slowly, in
which case the "peak-hourly retrieval" fee is lower. Or retrieve just a few
things (and stay under the free-retrieval maximum).

~~~
blackdanube
Do you show the free quota in the UI as well? Or is there any option to tell
it to always stay below the free quota?

------
stcredzero
The site seems to have gone down at the time of this writing. I am looking for
a backup alternative to Time Machine because it seems to have a memory leak.
The fraction of available memory just shrinks, even if I quit everything. I've
turned Time Machine backups off in System Preferences and rigged launched to
back up only 3 times a day, and now I'm no longer having to reboot every other
day.

A shame, because TM seemed to work perfectly in Snow Leopard.

~~~
sreitshamer
The site's up now! Sorry about that.

------
davidcollantes
Like many, I welcome the inclusion of Amazon Glacier.

I don't think that alone merited a major version bump[1], but understand that
the developer needs to feed his family, so will gladly pay for the upgrade.

[1] The ability to create the backup bucket, and not have it auto-create, for
example, would be something that I would consider a major version bump.

------
cynicalkane
Are the metadata stored in S3 or Glacier? Can you choose to recover only
particular files? Then you can use Glacier not only as a last resort backup,
but also as a defense against bit-rot if say one backup fails and the other is
corrupt.

~~~
sreitshamer
(I'm the guy behind Arq) The backup record data are stored in S3. Otherwise it
would take 4 hours just to look something up, which would be unusable. You can
choose to recover individual files, just as before.

------
cpeterso
As a longtime Arq user, I'm bummed there is no way to convert my existing Arq
S3 backups to Glacier without uploading everything again. I know this is a
limitation of Amazon's Glacier API, so maybe it will be possible in the
future..

~~~
gnyman
According to <http://aws.amazon.com/glacier/> it seems like Amazon will be
introducing this sometime in the future.

"In the coming months, Amazon Simple Storage Service (Amazon S3) plans to
introduce an option that will allow you to seamlessly move data between Amazon
S3 and Amazon Glacier using data lifecycle policies."

------
jimwalsh
Great work as always Arq team. More options are always great.

------
egonschiele
I'm excited about the new feature, but don't like having to pay $15 for it :/
It's a small price in the long run but I feel a bit cheated.

~~~
hemancuso
It's a feature whose sole purpose is to save you meaningful amounts of money.
Multiples of the upgrade cost. You feel cheated? Don't upgrade :)

------
alpb
Just figured out this is better than CrashPlan, now I'll use that for my 128
gb SSD. Great idea.

------
carterschonwald
the arq3 UI makes it a bit unclear when a glacier backup will happen and for
what data. (trying to discern that right now, and I cant!)

~~~
rb2k_
Arq will pop up a little dialog when adding a new folder.

------
aoe
Umm, so retrieving at 1MB/s is around $2/GB?

~~~
luser001
Transferring 1 MB/sec for 1 hour = 3.51 GB per hour

Assume you stored 1 TB (1024 GB) and need to retrieve all of it.

Free allowance per day = (1024 * 0.05)/30 = 1.70 GB/day

Free hourly transfer allowance = 1.70/24 = 0.07 GB/hour

Billable hourly transfer rate = 3.51 - 0.07 = 3.44 GB/hour

Retrieval fee = 3.44 * 720 * 0.01 = $24.76

Based on info here:
[https://aws.amazon.com/glacier/faqs/#How_much_data_can_I_ret...](https://aws.amazon.com/glacier/faqs/#How_much_data_can_I_retrieve_for_free)

~~~
DenisM
In addition to the $25 retrieval fee there will also be a $120 bandwidth fee,
so we're looking $145. And it will take 12 days to complete, at this speed.

Bumping up the speed by a factor of 10 will bump up the retrieval fee by the
same factor, so $250 + $120 = $370 total restore cost in a little over a day's
time.

Hm. I wonder if you can bill the insurance company for this in case of fire
etc.

Alternatively, it would be nice if Amazon allowed several accounts to pool
together their retrieval allowance - it's not likely that all of my friends
will have their house burn down at the same time.

------
danso
The OP quotes storing a terabyte a month as costing $10 on Glacier. I haven't
done mass backup to S3...but I was almost expecting this to be the cost for a
TB on S3, not on Glacier.

You can currently get a 3TB, USB 3.0 TB drive for as little as $120. At $10 a
month to store 1TB on Glacier, you'll surpass that cost in a year.

I know Glacier data is likely more safe than moving your files to an external
HD and then putting it in your safe deposit box...but the HD ends up being
cheaper in the medium-term and far more flexible and faster. It's unlikely
your house will burn down and all of your available backups...it's even less
likely that that situation will happen and that this last-resort external HD
will somehow also be lost, right?

~~~
luser001
Amazon strongly implies that your data is stored triple redundantly. You'll
have to buy more than one disk to get triple redundancy. Also, you'll need to
"scrub" the data regularly to detect bit rot (100 bytes per TB are expected to
go bad every year). And probably store the files with redundant coding to
protect against this bit rot.

Plug: I am working on a startup (submitted to this YC round!) to solve this
problem using user^Wcustomer-owned hardware for precisely the sort of reasons
you describe. I am looking for co-founders. If anybody wants to talk, email is
in my profile.

Plug #2: I wrote and submitted this article about Glacier yesterday but it
sank fast: [http://psranga.github.com/articles/possible-architecture-
of-...](http://psranga.github.com/articles/possible-architecture-of-amazon-
glacier.html) Email me if you want to talk about going up against an 800 lb
gorilla. :)

~~~
saidajigumi
One external online provider really only counts as "one copy", ever. This is
primarily because you cannot audit the ongoing storage architecture and
processes of any given provider. You're looking for SPOFs, not how many disks
may hold data replicas. One software error (or site/account hack) can wipe out
all of your data. Or an entirely out-of-band error occurs: the provider goes
belly-up.

Cloud storage is awesome in many ways. Yet it doesn't replace your backup
strategy, it merely complements it.

~~~
rsync
We regularly run an ad campaign on reddit discussing that very notion:

[http://www.reddit.com/comments/hg9oa/your_platform_is_on_aws...](http://www.reddit.com/comments/hg9oa/your_platform_is_on_aws_and_your_offsite_backups/?sort=old)

... that a single provider is really just a single "copy".

It is also the reason that we build 's3cmd' into our environment and so many
customers use it:

ssh user@rsync.net s3cmd put abc.txt s3://account/abc.txt

------
mitchi
I wouldn't trust Amazon Glacier. I'm guessing that they have a secret Glacier
datacenter in the Artic Pole. What happens when the glaciers melt? You are
done for.

~~~
hemancuso
I, too, would like to hear the OP's thoughts on Glacier durability in the
context of global warming. Doesn't seem good.

