

Rackspace hardware failure, all data lost. Oops... - nixle

Just a friendly reminder to not put too much trust in "the cloud", it was just our development server and we are rebuilding... but not with Rackspace. The thing is that this is the only message I have received about it. It's pretty much: "yeah, we lost all your data, good luck".<p>===========================<p>Hello,<p>This message is a follow-up to our previous emails. Despite the very best efforts of our engineering and datacenter operations teams, unfortunately the host machine that your cloud server xxxxx resides on was not able to be recovered. At this point, there is complete data loss on the server.<p>If you have taken a server image, your next step will be to build a new server from the server image. If you require the same IP address, please contact support prior to deleting the failed server from your account and we will assist you in sharing the IP to the new server. If you have not taken a server image, your next step will be to build a new server from a stock image and then upload data from your local backup.<p>We are extremely sorry for this inconvenience. Our Billing Team is going to process a xxxx credit for the server, equal to the monthly cost.<p>If you have any further questions, please contact a member of our support team by responding to this ticket, visiting us in live-chat, or by calling xxxxx or xxxxx<p>Best Regards, 
The Rackspace Cloud 
US Toll Free: xxxxxxxxx<p>======================
======
dholowiski
This all depends... were you paying for a managed server that they were
supposed to be backing up? If so, massive fail by rackspace. But if you were
supposed to be in charge of the backups, then don't blame rackspace.

------
facorreia
You can outsource your hardware, but you can't outsource your responsibility.
Backups and redundancy are your responsibility, not your service provider's.

~~~
freehunter
Another thing of note is your service agreement. Check what responsibilities
and restitution are present in the contract. If you need them to be liable for
damages beyond downtime, make sure it says that in the contract. If you need
them to failover, make sure it says that in the contract. If it doesn't,
well... it's pretty much: "yeah, we lost all your data, good luck".

~~~
toomuchtodo
I've done hosting for over a decade; no provider with a brain is ever liable
for anything more than what you've paid for.

~~~
freehunter
The company I work for has cloud hosting contracts which state if the service
goes down, they are liable for a certain percentage of lost revenue due to the
outage. This comes with a 100% uptime guarantee, including several failovers.
We pay dearly for it, but if you need something specific, you'll likely be
able to negotiate a contract based on what you need. If it's not in the
contract, you won't get it was my point.

~~~
toomuchtodo
Fair enough; but you're still going to limit your liability. Assume your
customer spends $10K/month with you; or $120K/year. Would you ever commit to
compensating them $1MM+ in the event of an outage? Of course not. It would be
business suicide.

If you want that level of uptime, you build out your own place (i.e. Amazon,
Facebook, Google). If you're not big enough to build your own, you live with
the possibility of downtime.

~~~
freehunter
True story. I took your comment two ways, and wrote my response based on the
wrong interpretation. For that I apologize. Of course we pay more than the
host could ever be expected to pay out. It's an insurance policy, when it
comes down to it. It's also less than we would have to pay to get our own
datacenter put up faster. As a stop-gap between the time our software went
live and the time when our datacenter will be operational, it's worth it. Time
to market was very important, moreso than the cost we pay up front. Losing
marketshare forever is worse than an $8b/yr company losing a few million
dollars for just one quarter. You and I are not disagreeing.

But this is all getting off topic. What I was originally addressing was the
OP's comment of "sorry, we lost your stuff, oh well". If you want that peace
of mind, you're going to be paying for it.

~~~
toomuchtodo
"But this is all getting off topic. What I was originally addressing was the
OP's comment of "sorry, we lost your stuff, oh well". If you want that peace
of mind, you're going to be paying for it."

Agreed.

------
jgeorge
Rage level depends on what you're paying for. I have a server at Rackspace,
it's my job to ensure backups, not theirs. That's the service level I pay for.
They offer a backup service through Cloud Files that puts your backup on a
different server. About once a month (which is all I need for the little usage
my server gets) I log in, click the button to do a Cloud Files backup, and I'm
done. Costs me about 60 cents a month for the file storage, pretty cheap for
an off-server backup. If the server holding my instance fails, it's a couple
of clicks and about 10min to restore it back.

I hate it for anyone who loses data, but if you're not paying them for a
backup service, it's an expensive lesson to learn.

------
hluska
No backups, hey? That's rough my friend and I feel bad for you. Best of luck
recovering and thanks for the reminder; I'm going to make sure that my backup
scripts work!!!

