

BuildYourVPS suffers massive data loss - MatthewPhillips

You are receiving this email because you have a VPS hosted through TOCICI's BuildYourVPS services.<p>We encountered a data loss issue on the storage array hosting your VPS environment, details of which follows.<p>Your VPS was previously hosted from a hardware RAID based data storage array which encountered a firmware level fault earlier today. Our server vendor (IBM) had apparently not seen this particular bug prior to today, which has led to hours of non-stop troubleshooting and diagnostics throughout today (tracked at: http://status.tocici.com ).<p>The effects of this issue are an inability to access any data stored on the inaccessible array: despite advanced troubleshooting alongside IBM engineers, we remain unable to access any data previously stored within your VPS.<p>IBM is continuing to work with us, including a rapid shipment of replacement equipment, and in-depth root-cause evaluation assistance and support. Although in the meantime, in order to re-establish your VPS environment as quickly as possible, we have re-designated your VPS's host node; defining one of our existing standby nodes. You should see a new VPS build email soon.<p>We share your frustration over this issue, especially with the sudden loss of multiple drives in a high-quality vendor's hardware based RAID...an incident that in nearly 20 years of doing business, we have never seen before. We are continuing to explore options, and we will keep you updated regarding any positive developments.<p>Sincerely,
TOCICI's Entire Support Team
======
MatthewPhillips
I received this email at 3:26, my VPS was wiped out. They built new ones, and
luckily I only had a hobby project static site sitting there.

This is pure speculation but I feel like BuildYourVPS is running too many VMs
on their servers. SSH has always felt sluggish. This is one of the downsides
to a cheap host.

I'd highly recommend prgmr.com (run by a fellow HNer), I've never had an issue
with their VPSes at all.

~~~
pestaa
From prgmr.com:

    
    
        ordering is disabled; I'm having some hardware issues
    

I remember having seen this, and it was a long time ago.

~~~
MatthewPhillips
I consider this to be a feature, rather than adding new customers that they
can't support fully, they turn off registration. Others might just stick new
customers on the old hardware.

~~~
pestaa
It might be a relieving feature for the insiders, but it unfortunately renders
your post to "I highly recommend these closed doors".

~~~
MatthewPhillips
I don't think the doors are closed permanently. I know you said you have seen
the message before but I think you've just checked at inconvenient times. I
added a new VPS just a month ago. If you need a VPS tomorrow then yeah I can
feel your pain.

------
jacknews
It's one of the downsides to hardware raid. IMHO it is a liability, and
obsolete.

~~~
pestaa
What are the possible alternatives?

~~~
stl
ZFS would be a good alternative.

~~~
Ecio78
They said that they had a "firmware level fault", this could happen also on a
card used by a ZFS storage, or you can have a bug in the ZFS code..

it would be nice to know which IBM storage they have 'cause I've just had a
bug in the battery firmware of Netapp storage on Easter (IBM rebrands Netapp
too)

