Hacker News new | comments | show | ask | jobs | submit login

> configured backups.

This part seems to be glossed over but is a HUGE issue.

It sounds like several companies have tried to pay the ransom with varying levels of success [0] ... why are they not just restoring from backup? I can only assume they don't have backups. (!)

What is their DR plan if the server dies? Or someone accidentally pushes code that messes up the contents of the DB? Or someone tries to drop the development database but oops: they didn't notice they were connected to the production server?

Even if you're using a hosted service, what if they go down? Get hacked? Lock you out because of a billing dispute/TOS violation/DMCA takedown/accident? Hired bad sysadmins that didn't do backups (correctly)?

Not having backups of your data is inexcusable and just reeks of utter incompetence, and has nothing to do with configuration defaults or documentation.

[0] https://krebsonsecurity.com/2017/01/extortionists-wipe-thous...

For small data volumes, MongoDB Inc. will actually back things up for you for $2.50/GB/mo storage (free bandwidth), and it's a live backup with multiple historical versions (an agent on your database machine tails the oplog, so you're at most a couple seconds out of date). And their free monitoring tier is great alongside it. I'm truly surprised that they don't sell it harder; it took me years of using Mongo before I knew this service existed.


If you're also hosting your data with them though, it's not a complete backup solution. You're still subject to essentially a single point of failure: if they are hacked, or suspend your account, get a DMCA notice against you, or are shut down by the SEC[1], both your live database and backups are subject to loss.

You still need to do external backups. You may have a lot of trust in the provider and these less frequently, but you should still do them.

[1] Had this happen to me once in the early 2000's: company I worked for had a dedicated server at a colo facility. After several days of them not responding to phone/email/etc, their answering machine was changed to a message saying the SEC had seized all assets and had all the owners 'under investigation' or something like that. We had external backups, but immediately took the latest stuff and got everything migrated to a new system in a new facility. Server stayed up for a few weeks after that, but then suddenly their whole IP space went offline. We never did get our server back.

I used to use Cloud Manager until they started charging a fee of I think $99 per server per month. I pay $150 for my hardware per month, so I stopped using their backup and monitoring tools.

Other than the cost, I recommend it for people who can afford it. Wonderful service that I was happy with for a long time.

Did the same, switched to Ansible for the Mongo deploy/management, and just use the free monitoring part of Cloud Manager.

Check out the ransoms being charged. It's .2 btc, or about $150 at current btc levels (which I acknowledge are unstable). That's pretty cheap. This is probably cheaper than having a single sysadmin spend a few hours restoring from backup just in raw labor time, not to mention everything else. Factor in the cost of downtime and the likelihood that you're going to lose some hours of user data since your last backup occurred, and the ransom is easily worthwhile to pay and get the server back.

I'm obviously not going to defend companies that don't have current backups (though this is practically everyone), and the importance of backups is always a great thing to emphasize, but in this case, the best option is to pay the ransom and get your stuff back.

You should read the article I linked though. After taking the data and replacing it with a README that says where to pay the ransom, the server is still left unsecured.. so someone else comes along, takes down that README and puts their own up with their own bitcoin address. Presumably if you pay them, you'll get back the next ransom note that says who to pay next, and so on, and as long as no one screwed up and everyone was honest about the ransom, after you've paid several people you'd eventually get your data back.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact