I think it was the 5.7.8 (Windows) release that introduced a bug that caused Arq to believe a budget of 0 should be enforced. This meant if you backed up to AWS (probably others too) it would delete ALL backup records except the most recent one (and then immediately cleanup the now unreferenced objects, so even the reflog feature can't help you)
I had only been using Arq for a few months so I didn't lose anything major, but if I had been keeping years of historical records, I would be furious.
This was fixed in the next release 5.7.13 ("Fixed an issue that could cause Arq to enforce a budget when no budget was configured.") but still - I thought a bigger deal should have been made of this.
Of course, either way of putting it would be engaging in lazy stereotypes about both platforms that haven't been accurate for years, if they ever really were; so let's not.
NB: I'm a very happy Arq user; I bought it because Time Machine crapped itself every couple months requiring me to recreate the backup (a pain in it's own right: 400g over wifi) and leading to a severe loss of confidence that backups were actually going to be there if I need them. I tracked it down to Time Machine gets really unhappy if you sleep your laptop while it's in the middle of a backup. However, having to check if Time Machine is running before closing the lid or being willing to run a 1-2% chance of corrupting the backup isn't really a workable state of affairs.
In a couple years, Arq hasn't eaten my backups. It's like Time Machine except written by competent people. This situation sucks but I'm sure Stefan will add enough tests.
On a digression, this is part of my increasing frustration with how shitty Apple sourced mac software is. This tiny company beats the pants off Time Machine, even including the recent screwup.
I imagine most or all of us with that sentiment did not suffer any data loss or rather did not need to do a retrieval and discovered data loss during that time. So much easier for us to write it off as a non issue. Where is if it affected us I imagine we'd all be quite a bit more upset.
Though most people use common alternatives such as Time Machine which every few months runs into data issues and far less transparency. Stefan could have quietly updated with a fix, and waited to see if anyone discovered the issue and then only announced the issue if that occurred. But instead he proactive announced it. So bravo. And I'm sure he will instill new procedures to hopefully avoid that in the future, but of course with any software there is always room for error no matter how much testing you do.
Ont this note, I've thought about something at times: If there was one feature that I would like to see is options to verify backups. Beyond the possibility of Arq having a fault, it's also very possible that a backup service has silently failed. Either, the files are corrupted or the service lies that the files exist but you can't actually download them. I would appreciate the ability to have files pulled (either periodic random or marked files) pulled and compared to local files to verify the backup. I would also want to cap the max files/bytes pulled in a month so that I don't burn too much money.
I know it took guts to make the post. You did well.
And BTW, I am a happy Arq user, I've been using it on multiple machines for the past several years. It is the ONLY backup solution out there that:
* encrypts my data without me having to supply the decryption key to a third party when decrypting,
* decouples backup and encryption from a particular storage provider,
* makes me independent of ebbs and flows of storage providers by supporting pretty much every one out there.
I've learned to value independence over cheapness, so I stick with Arq. Plus, it's been working really well over the years.
I love using Arq on my MacBook Pro, but since they won't release a Linux client I use Duplicati on Ubuntu. I like it a lot.
In the long run (Borg 1.2 - 1.3) we'll add an interface to Borg to make it storage-independent ("repository drivers") in a secure way (this is already sketched out, just not enough manpower in the project to implement it all right now. Finishing 1.1 has top priority and received a bit of funding).
Adding support for S3 and $threeDozenOtherAPIs directly to Borg is generally speaking unlikely.
PS: To-be-released Borg 1.1 will optionally get rid of cache syncs for "borg create".
How do people feel about killswitches connected to C&C servers so that if one releases broken software, one can block its use, or force an update? I wouldn't be very upset if it were just a flag you could turn off to prevent killswitch enforcement.
This only increases my trust in Arq. Every program will have terrible bugs once in a while, even backup software. This shows that we can trust Haystack software to inform us, rather than sweeping things under the rug.
And as always: make multiple backups to different locations.
Beyond randomly checking a time in the past and trying to download a file, is there a way I can tell which / how many files are now missing?
Shit happens, but that was particularly egregious. This is why you should have multiple backup methods in place, and if you've never restored one, you don't have a backup.
I'm on a Mac - I use Dropbox (with permanent version history turned on) for common files, and Time Machine to a Drobo for short-term versioned/incremental backups (short-term because the backup invariably gets corrupted every year or so and I have to start over). (I also currently use Crashplan but am thinking of switching to Arq since Crashplan has very slow upload speeds - slower than my ISPs upload speeds.)
I'm also figuring out a way to remotely have a mirrored boot drive for my Macbook but haven't quite gotten there.
Backblaze runs all the time on _everything_ (some 4TB), and I have Arq running nightly on my most important stuff (photos, documents, music, recordings) to AWS as well as Amazon Cloud Drive.
Since 1980, I have learned to keep multiple backups, using multiple independent software, stored in multiple locations, on multiple types of media, overseen by multiple people.
An overview of my personal backup system for my primary system a MacBook Pro:
* Boot able mirror to remote hard drive via Carbon Copy Cloner
* Data backup using Arq to home file server with RAID, and Google Cloud (previously to Amazon and Microsoft)
* Data backup using Backblaze
* Data sync via Resilio to home server
* Code stored in Git
* Some stuff in DropBox primarily sync with iOS apps
* FastMail IMAP email synced to a local backup store
* I just stopped use of CrashPlan after 9 years
I have other backups going for the home file server.
And of course, my servers for work have a whole other multilayered backup system.
"If we could, we’d make backup copies of our valuable data on clay tablets" https://landing.google.com/sre/book/chapters/data-integrity....