Comcast: Our costs just went down. Effective immediately, we are adding a new cash management fee to your bill to cover our costs of handling all this new cash.
Below scale, cash management is thrown in just to get people's business, but at scale, it costs what it costs. Walmart, which has substantially more problems than you on that score, reportedly spends millions of dollars every year just dealing with pennies.
I can't think of TOO many more first world problems than that, to be honest...
Of course, if I ever deposited cash I would be on a different banking plan; but it's not entirely unheard-of.
Colin, do you have a contingency plan in place if you are not available?
I don't recall if this was on Tarsnap's website when I had first checked, but it might be worth reporting the average compression rate to make more visible to prospective customers what the Amazon S3 storage portion of the pricing is.
The whole point to tarsnap is to be able to safely encrypt any and all of your data, including compressed data. We wouldn't expect that a .tar.gz being backed up is somehow "less secure" than the .tar file, we'd demand the same security for both.
Compression becomes a problem when it can be used as an "oracle" into the key used for a given stream of ciphertext. The reason TLS is susceptible is because the attacker can MITM and control at least some aspects of the plaintext or shared client-server state, and iterate repeatedly to refine their guesses.
These issues simply don't apply in the same way to backing up files.
I sure there are still theoretical issues that would need to be worked through in deciding how you'd do something like this, issues which could be better explained by any of the many crypto types who hang out here. But don't cargo cult your treatment of crypto.
And it de-duplicates as well.
MAC, decrypt, decompress.... always^9 in that order.
However if my choice for the latter is between a sole trader and an organisation with 20-30 employees and multiple directors that obviously is the more sensible choice. No matter how much more you like the tools or respect the owner with something as important as backups you need to go with the most reliable option.
Sure, I could use tarsnap and another file store for three backup locations (which may be something I should do) but then the argument is why not go with two companies that have a bus factor of more than 1?
Again sorry Colin for this but there is a 1/500,000 chance he will be struck by lightning this year . Amazon 3s (the backend of tarsnap) has a durability of 99.999999999%  and so you are 200 thousand times more likely to lose your backup because of a lightning strike than due to a corruption on s3.
I have a secondary emergency backup in a bank vault, so even in the unlikely event that tarsnap were to crash and burn at the same time my disk fails, I still have another backup, it's just 1-2 months old.
Honestly, I'm not sure I would be a tarsnap customer if it were a larger operation where I could not gauge the trustworthiness and skill of all the involved parties.
(God, that's a strange sentence to write. Sorry about that, Colin.)
I can't say that I spend a lot of time keeping up-to-date on the project leaders' obituaries for services I use. I don't imagine I'm unusual in this regard :)
Edit: This is just a rhetorical question, not a demand for explanation :) - just saying that being hit by a bus does not automatically mean that users are shifting off your service in 24 hours.
There'd be a cash price and everything. I am sure several people worked on logos - some excellent, some less so. I'm frankly surprised to see that this is the result:
Really? Did some of us just waste our time so that Colin could have his 8 year old nephew whip up a blurry logo? It's ofcourse his right to do so, but it does make me wonder.
"During this time, 83 people submitted over 100 designs . . . The winner, as promised, received $500; I decided to create a second prize of $100 for the designer of the other logo, in part as an apology for the time I took up asking him for revisions"
There's also a square version here: https://www.freebsdfoundation.org/donate/sponsors
I agree the blurriness could be improved, though.
As for people entering a contest... they can't all be winners.
You'll have to experiment with the dpi until the logo comes out the right size. The dpi doesn't have to be an integer.
No, seriously. What do most people use it for? Simply creating a daily backup of their hard drives? Also, are there any business users who use it to backup an entire organisation's systems?
Btw, more on-topic, I'm reminded of this quote, which I love, from Jeff Bezos: "There are two kinds of companies: Those that work to try to charge more and those that work to charge less. We will be the second."
I've always thought that was a profound sentiment, and I've always wondered if, in the long run, that's the way to make a business survive.
This is all anecdotal, but I think most people are at least somewhat selective in what they back up. In my case, I have some servers where I back up /, but on my laptop I only back up my home directory because I know if my laptop dies I'll be reinstalling FreeBSD from scratch anyway.
Also, are there any business users who use it to backup an entire organisation's systems?
I think so, but I'm not going to name names. Maybe some Tarsnap customers will reply here.
this quote, which I love, from Jeff Bezos: "There are two kinds of companies: Those that work to try to charge more and those that work to charge less. We will be the second."
Yes, I found that quite inspiring too. And it has certainly worked well for Amazon.
It's worked well for Wal-mart too, but I don't think anyone would praise them for doing so.
Stripe has been a happy Tarsnap customer for several years. It's robust and thoughtfully designed, Colin has provided great support, and "backups for the truly paranoid" is a pitch that very much resonates with us.
I use https://github.com/Grive/grive to sync Google Drive to an encrypted local directory on my workstation at home. This is done daily.
Then I use TarSnap to backup that local directory.
Effectively I create an on-site copy of important docs from Google Drive, and then use TarSnap to create a secure and trusted off-site backup of those important docs.
Oh, and we also use TarSnap to backup our company (product) databases once a day too (though interim backups are stored closer to the servers too).
Our entire organisation is handled thus:
1) Systems and code via Github, and pulled often to one place (that local workstation) and then backed up.
2) Data dumps pulled locally and sent over to TarSnap of all product/customer data and all company files.
And the only on-faith thing is file attachments in the customer data:
3) Files via Amazon S3, trusting the durability of S3 and security of our interface to it.
This is all disaster recovery stuff. Secure, trusted backups.
Thats works well when your prices are based on 3rd party prices and you can optimize your business to keep your margin. But, for instance, a service oriented company, or a freelancer, will usually try to improve his/its skills and experience to charge more because the endresult is worth it.
You can make money by working more effectively and efficiently while still charging less for the same "product".
If you can do the task in a quarter of the time and charge half as much for it then you can double your bottom line.
¹ (assuming 16 months between major versions, as in 3 → 4)
This really is Tarsnap's key differentiator (for me).
It's also worth noting that Arq does have command line functionality (on OSX) as well as an open source CLI restore tool.
So, $30/Year buys you only 10GB-month of storage on Tarsnap.
You could distribute that equally throughout the year, using 10GB-month per month, yes. Or you could use 1GB-month in the first month, 2GB-month in the second, etc, as your storage needs grew.
TARGETS="/etc /home /var /root /srv"
tarsnap -v -c -f "$ARCHIVE" $TARGETS
 https://github.com/pronoiac/tarsnap-cron - which might be helpful to someone else. It splits up the archiving and the pruning of old archives, so you can do those with different keys. This part is as yet untested. It probably requires an tarsnap fsck if you do those on different systems. I was going to forgo the plug, but it might be helpful, and I'm about to get some sleep, so I'll probably miss the best time to contribute to the discussion.
That depends on your customer and what your product positioning is. If, for example, you run a discount store, a la Amazon or Wal-Mart, then that advice will work well for you. If, however, you are running a company that sells products that are positioned as premium, lowering your prices can hurt you.
Cutting costs on premium products immediately changes customer perception, such as when Starter was purchased by Wal-Mart or Rock & Republic was purchased by Kohl's.
That is a daily snapshot of my /etc, important dotfiles, and game saves. I haven't needed it yet, but I'm sure I will be glad when I do.
EDIT: I just have a simple script in my cron.daily that backs up files and folders that I tell it to.
Weekly backups of my VPSs, daily backups of my email and Tiny Tiny RSS database. I'm probably worthless as a customer :|
I find that the customers who are paying $0.01/month often provide the most enthusiastic word-of-mouth advertising. You're not useless at all, no matter how little you have stored. ;-)
[I haven't looked up your account, so I don't know if you're over or under $0.01/month, but the precise number really doesn't matter.]
I use it for daily DB backups for 2 small website databases. It's cost me about 2 cents for 4 months.
It's only about 2MB per day to upload.
I have a daily cron that does the backup and deletes all but the latest two images.
Quality of service, along with Colin's amazing pricing, make it an easy win for me. I never feel like I need to stop and waste time thinking about other options.
If he disappears, suddenly my backups aren't available, AND my drive blows up, all at the same time, then I'll be sorry I guess.
Tarsnaps security is based on client-side encryption. They cannot read the plaintext. Therefore if more than one person uploads the same file, they will have two different encrypted versions of the same file, which can not be deduplicated.
You could use convergent encryption, but then users can query each others files.
In the presence of competition this is the case. Fat margins attract competitors.
To go just by the stated pricing of Tarsnap, this would cost me $25 per month for the storage. But then I also see mention of users with terabytes worth of archives who pay less than $10 a month. I read the FAQ entry which explains how this happens, but that does not really tell me whether I can hope for such savings when it comes to photos. Do photo collections (raw/jpeg/both) "shrink" significantly from the deduplication and compression of Tarsnap? I think they don't (and that the savings apply to incremental backups), but it would be great if you could make this clear. Thank you!
If you are going to store 100GB of photos with Tarsnap, I'd guess it would be close to 25$ as you said. If you just want your photo collection for disaster recovery, you could check out Glacier instead, which is a lot cheaper.
Great, that's out of the way. Storing and archiving photos & videos is near and dear to my heart  and I believe that cloud storage is one piece of answering yes to the question of "will I have my photos in 50 years?". My entire Shuttleworth Fellowship is based on this.
Here's my $.02.
Cost - we haven't yet but have all the pieces to use Amazon Glacier for storage of high resolution originals. That means storing 100 GB will cost you $1 / month. There's additional costs to keep thumbnails in S3 for immediate access -- my estimate is <$3 all inclusive for that 100GB.
Ownership / Portability - a big part of my fellowship is to see how a hosted service (ala Flickr) can provide 100% ownership and portability. The solution is to let users (optionally) bring their own storage. So you can use the Trovebox software connected to your own Glacier and S3 bucket.
Functionality - I think organization, viewing, sharing and archiving should be merged together. Instead of having your sharable photos in one place and your archives in another --- why aren't they combined?
Raw - we support RAW and do conversion to JPEG.
Open Source - Check .
Mobile - Check    .
API - Check .
This is the fundamental problem I've always seen with cloud storage for photo/video pros (or hobbyists): they need long-term storage but the bill just keeps growing. Should they be expected to pay $1k/mo after they've been in business 10 years?
On a separate note, how are you doing RAW <-> JPEG conversion on the server?
The only hope is that cloud storage goes down over those same 10 years at a rate which makes it continually affordable. But it won't always be a fixed cost since the number of photos keeps going up.
The RAW -> JPEG conversion is done using ufraw . We originally tried extracting the thumbnails so the JPEGs would use conversion settings from the camera but most of the thumbnails are too small to do anything useful with.
How would one go about making sure that when a server is compromised, the malicious attacker wouldn't be able to delete all the tarsnap archives for that machine? Since the tarsnap.key is stored on the server itself and that's all you need to delete archives as well. Of course, you're already properly effed when an attacker has root access to the machine, but offsite backups should still be safe imho.
That's why on some of my servers I have a 'pull'-backup strategy in place, where a remote server would connect to the machine to be backupped and pull a backup, so in an event of that the server would be compromised no backups could be deleted. Is this something that can be achieved with Tarsnap as well?
That's certainly part of it. Let's face it, a lot of people use Tarsnap because of my reputation; I'd like to have Tarsnap contribute to my reputation rather than merely taking advantage of it.
I'm pretty sure I've run across examples in 19th-century American novels of this sort of "fair price with a modest profit" ethos.
I guess I really am a bit anachronistic...
(stolen from tom)
He could still pay himself a fair salary but he'd avoid any tax liability for the business and I guess he could accept donations too?
I don't know much about it, but if your only goal is a fair salary I've always assumed a non profit business structure would make sense? Can someone more knowledgable chime in?
Crashplan, Spideroak and Wuala (for 100GB and under) are a bit cheaper ( http://skeptu.com/tarsnap/100gb ).
It has a nice CLI, however.
CrashPlan and co. are somehow secure, but still not really open about their crypto (and some of them use quite weird one).
I'd much rather trust my backups to tarsnap, which is not in beta. Furthermore, and perhaps even more importantly, tarsnap offers support which duplicity does not. When my hard drives decide to die, I really want someone who I can contact if there are any issues restoring from backups.
Unfortunately, it's barely usable due to recovery issues. You can't mount CrashPlan as a filesystem (well, without ton of reverse engineering), so that's not an option unless you're satisfied with all-or-nothing recovery without possibility to pick and restore just certain files of interest.
You could probably also hack up something to figure out locally what encrypted filename corresponds to what regular filename and go fish for the encrypted filename in CrashPlan. It will probably be clunky though.
We were talking about eCryptfs/encFS-encrypted copy, where filenames are (usually) encrypted. That means navigating around names like `l00Dqf,A49VqDd8AveLMrbBE` or `qR,bmE-73cA2H6wOxZxlKSwD`.
The proxy is run locally and all encryption happens locally, so it's end-to-end encrypted.
$ tarsnap --print-stats
Total size Compressed size
All archives 535 GB 233 GB
(unique data) 1.2 GB 438 MB
Deduplication across users isn't possible due to encryption, but it is done locally on your own data, so it is true that you only pay for unique and compressed data.
That's really nice. I'd rather have fair public utility rather than business for the sake of concentrating money.
I won't think twice about it the day I need to sign-up for tarsnap.
perhaps at some point he'll, say, marry, have kids, and want some more cash and financial security.
I am currently single, but if/when I find the right girl I don't expect finances to be the limiting factor.
Nothing wrong with that really, but if you really want to strictly price it as a utility there needs to be some form of return on equity. Even amazon generates some free cash flow, they've just been reinvesting it all so not generating any income.
I'm just going to assume that Colin is smart enough that he knows how to price his service. After all, he's the only one with the data to really know tarsnaps financial situation.
Been looking at tarsnap for a while but never gotten around to actually try it and my quick naive calculations for backing up my ~ makes it sound too expensive for me even though it's probably the best alternative i've seen so far.
''Don't really create an archive; just simulate doing so. The list of paths added to an archive (if the -v option is used) and statistics printed (if the --print-stats option is used) will be identical to if tarsnap is run without the --dry-run option.''
"we are thus launching a new price layer at .25, new users will be automatically enrolled in this one and you can switch by doing X"
This way people who want to actually donate more, or do not have enough of a preference would still give you more money.
(Of course loyalty penalties are commonplace -- from phone companies, car insurers, etc. -- but my mental model of cperciva says he probably doesn't like them.)
Exactly true. I was looking at that comment trying to put my finger on why I hated the idea so much, and you're right: It's because it would be a loyalty penalty. I want to be fair to everyone.
But for most personal uses, a Glacier-backed variant of tarsnap would be extremely appealing.
Add any minimal restoring cost and glacier costs about the same as regular storage -- with a lot less convenience. I honestly can't think of a use case for something like glacier, where storage is cheap but reading/writing is expensive.
The one big downside to the "cheap backup, expensive restore" approach is that it discourages testing your restore.
PS. Is there any future plans to make restore a little bit faster? It's been a while since I restored some files, but the process was so slow that I thought there was a connection problem, then I googled and found out that it's okay to be slow :-)
Are there any better tools out there to schedule backups and purge old backups? I'd be happy with a simple tool which just keeps 7 daily backups and that's it.
What's the second-most-critical bug Tarsnap has ever experienced? Just curious.
Fortunately this was a new feature and I was able to identify which people had used it (since it relies on server-side functionality and I had good logs) so I could email all the potentially-affected users to warn them.
So, Tarsnap uses a "chunkification cache" to speed up archiving; when it chews through a file and splits it into chunks, it records "file X was N bytes, had inode #I, and was last modified at time T, and here's the chunks it was split into". The next time it sees the file, it starts by stat()ing the file and if those parameters match it reuses the chunk list rather than reading the file from disk and splitting it into chunks again.
With checkpointing enabled, if a checkpoint occurred in the middle of a file a truncated entry (to be specific, the chunk list for the portion of the file which had been processed) would be stored in the chunkification cache. If a later tarsnap process read that it was possible that it would archive the file wrong (I think it would be truncated, but I'm not absolutely certain right now). The fix was simply to not use an entry from the chunkification cache if it wasn't internally consistent.