Consider this. To store 500 GB in Glacier it would cost be $5/month and I would have to do:
- Use a tool to manage my uploads, or write my own thing, or use the CLI or Web UI
- I get charged money for downloading content out. So checking that my backup works costs more than the backup. And it takes longer.
- Manually manage what has been backed up, and figure out the differences to get any kind of syncing or incremental backups.
- Have no idea about how Glacier data is actually stored or how redundant it actually is. Amazon will not talk about this publicly.
Or... I could spend $5/month on Backblaze where I could:
- Store as much data as I want
- Access it at any time, instantly, from any device, for free
- Get free, supported, automatic, "only upload the diffs" software to keep everything in sync
- Know exactly how my data is stored because Backblaze is super transparent about their storage pods, the harddrives, and the reliability of the various components. They even open sourced there hardware designs.
Glacier is neither an easy nor cheap backup solution for the end-user. Seriously, for an end-user computer just go get Backblaze, or Crashplan, or anything else.
(For businesses that already store things in S3, yet, I can see an advantage to "aging" data into Glacier. But this is a rather specific use case)
I'm on OSX. I was a long time CrashPlan user until on multiple occassions my stored data would simply vanish (e.g. yesterday it was 300GB, today it's 3GB and everything needs to be forcefully reuploaded for the next week). Spent quite a bit of time with support, sending logs and such, with no real resolution. And only because I happened to look at the UI was I aware of the issue. That didn't leave me comfortable at all, given I normally didn't pay attention once it was running.
I then looked at BackBlaze briefly. Same class of product as CrashPlan (set and forget), but I immediately ran into bugs (submitted, confirmed by them), was put off, still raw from CrashPlan... Not a fair assessment to be sure.
Landed on Arq. It's not without its annoyances, but it does do the job well enough, including the recovery process from Glacier which is pretty tedious. I like that Arq can easily map backups sets to different providers, like putting frequently changing stuff that I do restore as a convenience into S3 or Google Drive, instead of Glacier. My most important (smallish) files live on multiple services.
On a more general level, I like that I'm deciding exactly what I'm willing to pay for and how it's managed. I get the AWS bills. If I want infinite backups on this set, and the just the last week on a different set, no problem. The tool does my bidding, and does it smartly (deduplication, etc.)
Lastly, the Arq author is accessible. The whole experience has brought me from skeptical to fairly satisfied.
Also, Crashplan never deletes anything while Backblaze, last I checked, requires something like an external HD to be connected once every 30 days. A true backup, not offsite storage, in a sense.
It also has a family plan which is great, allowing 5 machines for U$ 13/month I think.
Yes, Backblaze's "window" is limited to 30 days. Crashplan may delete files but it's up to you:
"Backblaze will keep versions of a file that changes for up to 30 days.
However, Backblaze is not designed as an additional storage system when
you run out of space. Backblaze mirrors your drive. If you delete your
data, it will be deleted from Backblaze after 30 days." [1]
"If you delete files from your system, they remain backed up and in your
backup archive forever, as long as: 1. The files remain selected in your
backup file selection. 2. Your 'Remove deleted files' setting is set to
never." [2]
Backblaze provides a mirror of your running system with a 30 day max limit on diffs; if you discover a file disappeared or became corrupted more than 30 days in the past, Backblaze can't help you.
You can download 1GB/month from Glacier for free or you can transfer any amount from Glacier to an EC2 instance in the same region; one or both should be sufficient to do your own integrity testing.
Anyway, Backblaze is a backup service, Glacier is not. Glacier is a storage service you can use with any backup software written to be compatible with Glacier's API. Backup management including diffs is the backup software's job, not Glacier's. Glacier's job is to be very durable storage; I also wish Amazon would say more about their storage media but it's generally understood to be heterogeneous with multiple redundant copies of all data stored.
> or you can transfer any amount from Glacier to an EC2 instance in the same region
Note that it's only the transfer that is free. Telling Glacier to retrieve something, even from EC2, comes with all the same fees as if you did it from your own machine.
Glacier retrievals (separate from data transfers out) can also be effectively free but it can be complex. This complexity is a significant downside to Glacier.
"Backblaze is a backup service, Glacier is not. Glacier is a storage service"
This was my original point, and thank you for stating it so clearly. My concern with the article was that is was saying "Here is a storage service, and here is what I do to use it as a backup, isn't that cheap and easy!"
If you want a backup service, you are often better off purchasing a backup service instead of trying to roll your own on top of someone else's storage service. YMMV
I don't understand your claim about redundancy. Amazon says that Glacier "is designed to provide average annual durability of 99.999999999% for an archive", https://aws.amazon.com/glacier/details/#durability. This seems to be the same as for S3 objects. Furthermore, they say that "the service redundantly stores data in multiple facilities and on multiple devices within each facility".
So multiple facilities and multiple devices on the same facility (I guess that means at minimum 4 copies), and a calculated annual durability. Not so secret?
What are Backblaze's (and Crashplan's) redundancy policies?
I tried Backblaze a few years ago and it choked due to the number of files I had (millions, IIRC). Crashplan was going to take months for the initial sync. Have either improved?
I'm currently using a Mac OS X app called Arq, which offers similar features (incremental backups, end-to-end encryption), but backs up directly to Glacier/S3 (or SFTP, or Dreamhost/Google Drive/Google Cloud Storage/etc). It works well.
I prefer backup software that uses open protocols with commodity/"dumb" file storage providers. Given client-side encryption, there's not a whole lot of value to be added by middlemen.
Ideally, glacier is your backup of your backup. You only fall to it if your tapes fail, your disk-to-disk blows up, your datacenter melts in a fire, crashplan/backblaze go out of business.
For Crashplan an Backblaze, you are using their specific client, so if it doesn't suit your needs (retention schedules, file include/exclude control, multi tiered backups, special [i.e., database] file handling) you are out of luck.
So the ideal use of Glacier is to use a backup client that fits your needs, with S3/Glacier support built in. Or use an online backup provider that uses S3/Glacier as a backend to their product.
- Use a tool to manage my uploads, or write my own thing, or use the CLI or Web UI
- I get charged money for downloading content out. So checking that my backup works costs more than the backup. And it takes longer.
- Manually manage what has been backed up, and figure out the differences to get any kind of syncing or incremental backups.
- Have no idea about how Glacier data is actually stored or how redundant it actually is. Amazon will not talk about this publicly.
Or... I could spend $5/month on Backblaze where I could:
- Store as much data as I want
- Access it at any time, instantly, from any device, for free
- Get free, supported, automatic, "only upload the diffs" software to keep everything in sync
- Know exactly how my data is stored because Backblaze is super transparent about their storage pods, the harddrives, and the reliability of the various components. They even open sourced there hardware designs.
Glacier is neither an easy nor cheap backup solution for the end-user. Seriously, for an end-user computer just go get Backblaze, or Crashplan, or anything else.
(For businesses that already store things in S3, yet, I can see an advantage to "aging" data into Glacier. But this is a rather specific use case)