Hacker News new | comments | show | ask | jobs | submit login
Stop Using Digital Ocean Now: The Aftermath (serdardogruyol.com)
84 points by sdogruyol 1537 days ago | hide | past | web | favorite | 96 comments



Sorry but I must be blunt... What do people expect for < $8/mo service? Should DO spend a few man hours to fix the issue at hand (abuse or not) and very much burn the 7 months worth of service income from this account? This is an unreasonable expectation.

Something has to give. Yes we've all built and sold products and believe in providing an impeccable service worth far greater than the sum of its parts. Because we're in it to please everyone and build a reputation. But hosting is different. There are real costs for not taking action (upstream null routing, blacklisting, chargeback fees, fraud, abuse, etc).

I'm definitely playing devil's advocate and yes I go above and beyond for my clients. Never have I kicked a client to the curb for abuse they haven't originated. But DO is unmanaged and acquires clients by the shovel. Something has to give.

For the record, I did publicly state here in HN that DO's business plan doesn't add up and this is one of the side effects.


I expected a budget service, but they've been anything but that. Their support staff aren't just reading off a script, their instances are fast and downtime non existent. In this case the user fucked up and was made part of a botnet, and DO were forced to protect their network. I'm happier running my services on DO than on any other host.


This.

It was the same thing the budget dedicated providers did before 'cloud' and 'vps' became popular/viable technologies. I do think DO's business plan does make sense tho, many of those budget providers are large, profitable business today.


Abuse is expensive for a cloud service provider. Being thorough about it helps keep prices down.


On the other hand, here's my experience with Linode.

Another host company claimed that one of my machines was doing port scan on their network. Linode opened a ticket and preemptively blocked all outgoing connections to the SSH port from my machine.

I had enough time to see what's going on and chatted with a very responsive support. The aftermath is that I moved all my data to a new linode, waited to for the DNS propagation and killed the old linode. No service disruption and no all-nighters.

I seriously can't recommend they enough. And yes, I'm aware of the security problems they had months ago, but I bet that they don't want any more damage to their brand and they're working very hard to no let that happen again.


Not all VPS service providers are treated alike. Linode is probably a mid-tier VPS. Their cheapest plan is $20, which is 4x more expensive than Digital Ocean's cheapest plan of $5 a month. I would guess that most of the ultra cheap VPS plans probably won't come with a network engineer to help you.


Agreed. If I'm trying something new, or just playing with, I could try a new and cheaper host ($5/m is less than a starbucks coffee where I live).

But if I'm running something that pays me money, there's no doubt that I'll choose the best provider. And the best provider, for my requirements, is only $15 more expensive.


> The things which i still don’t understand.... Is my privacy is more important than my user experience or happiness of the service ? Even if i want them to tell what really happened ?

Well, yes. It is completely appropriate that they aren't going to discuss your logs and other private details in public without your permission. You don't understand that, for real?

They have no obligation to discuss your case in public at all, although it might be wise for them to do so and explain themselves, if you are generating lots of bad feelings for them.

(They do, I'd agree, have an obligation to discuss your case with you and tell you why they closed your account. It's not really clear to me if they did so; it kind of seems like they did so, something to do with DDoS.)


OP, can you please confirm whether or not you made this post quoting a reply from Digital Ocean in the previous thread, as [1] claims you did?

Now i've received an answer from DO. I seriously dont know how i did a DDos.

Here it's.

Greetings,

Based upon the tcpdump results, I have again confirmed that your droplet was indeed performing a Denial of Service attack.

With this information, we are unable to restore services to your account.

If this is true, it is disingenuous not to mention this reply in your post.

[1]: https://news.ycombinator.com/item?id=6439501


The thing is that they wanted to stay private when i posted this so i immediately removed and now waiting for them. By the way i received that response after 2 hours of the topic opening. They weren't answering me at all before HN post


So did your server get owned? Or was the DDOS attack a result of a bug in your code?

Also, why do you delete your posts on HN? Do you only keep posts that make your viewpoint look favorable?


I don't know the first question's answer and that's what i am trying to learn. Second it's my first time participating in a HN topic that much i didn't know the etiquette here sorry for that.


> I don't know the first question's answer and that's what i am trying to learn.

Well, if you don't claim to know all of the details then I'd say it's a bit disingenuous to make a blanket statement like "STOP USING X SERVICE". I thought the tech community was better than that...


I'm guessing that most people who run a server may not even realize when they get hacked. These people (you included) probably should not run their own servers and stick to PaaS solutions like Heroku or Google App Engine. It happens all the time to guys who think they can install & maintain Wordpress themselves.

You probably should have analyzed the issue before making a blog post about it. I have had servers hacked into in the past, but I wasn't about to defame a company over my own mistake of securing the server.


You miss the point here. It's not about getting hacked or so. It's their way of handling it. Like i said they kill it first and then tell you the reason why. What's the point in it ?


The point is that your system may be actively attacking another system, and it's their responsibility to immediately stop the attack first and then contact you after.

If they don't do this, then they run the risk of having their netblock(s) blackholed by upstream providers or other networks, which is bad for all of their customers.

They aren't responsible for making sure your system is secure, you are. You're a sysadmin now; it's not just a toy, there's responsibility too.


My guess is that Digital Ocean is the first VPS you've ever used. I don't know of any provider that will wait 10 hours for you to respond while the server sends out 1Gbps traffic. If they let the server continue running, the bandwidth costs would probably be more than 100x your monthly subscription cost. The VPS providers that don't shutdown your server will usually just bill you the bandwidth costs and you won't notice your server got hacked until you see your $800 bandwidth bill.

Killing it first is the proper course of action here. I understand you're new, so there's always a first time. I just disagree that you have to make an inflammatory blog post about it.


As a customer of DO, I'm glad they turned you off. Your ignorance was negatively affecting my service.


His server was probably broken into, someone used it for DDOS'ing, DO shut it down and customer has no clue what's going on.


Hi, this is Ben, CEO and Co-Founder of DigitalOcean, we have received the document and will discuss the matter publicly.

-----

All times are UTC.

Our monitoring picked up a malicious UDP traffic pattern on 2013-09-08 00:58:23. A ticket was then opened with the customer at : 2013-09-08 01:05:55 roughly 7 minutes later.

The customer informed us that it was a script that was crawling in the background.

We informed the customer that it may be a good idea to check through the virtual server to see if there were any signs of a compromise just in case.

The droplet was unlocked at this time.

A second UDP pattern was detected on 2013-09-24 12:27:09 and a ticket was opened 2013-09-24 12:27:14 to request more information from the customer.

Because this was already a second occurrence we had to do a more thorough follow up. Discussing the matter with the customer, he informed us that it was a mysql db dump script that was pushing data to dropbox.

He provided us a link to a github project that he wrote, we asked further questions. Specifically if you are writing a mysql dump remotely why are the packets being sent as UDP? Additionally if the final destination is dropbox that would be an SSL encrypted connection and again why would that transfer go over UDP?

We reviewed the code of the dump-to-cloud project and it was using the dropbox sdk, here is where the file transfer is initiated:

   def upload_file(file_name)

        client = DropboxClient.new(@access_token)

        file = open(file_name)

        puts 'Uploading file!! Please wait.'

        response = client.put_file("/#{file_name}", file)

        puts "uploaded:", response.inspect

    end
From the dropbox SDK here is where it sets the destination for the file transfer:

   def build_url(url, params=nil, content_server=false) # :nodoc:

        port = 443

        host = content_server ? Dropbox::API_CONTENT_SERVER : Dropbox::API_SERVER

        versioned_url = "/#{Dropbox::API_VERSION}#{url}"



        target = URI::Generic.new("https", nil, host, port, nil, versioned_url, nil, nil, nil)



        #add a locale param if we have one

        #initialize a params object is we don't have one

        if @locale

            (params ||= {})['locale']=@locale

        end



        if params

            target.query = params.collect {|k,v|

                CGI.escape(k) + "=" + CGI.escape(v)

            }.join("&")

        end



        target.to_s

    end


The code that actually transfers the file from the dropbox sdk:

    def do_put(url, headers=nil, body=nil)  # :nodoc:

        assert_authorized

        uri = URI.parse(url)

        do_http_with_body(uri, Net::HTTP::Put.new(uri.request_uri, headers), body)

    end

The file is transferred via HTTPS since it is going to a secure service and HTTPS would rely on TCP for the data transfer, again to ensure that all packets are delivered.

Given that it was the second incident that a UDP traffic pattern was observed in less than 30 days and that the information the customer provided regarding the traffic did not match up, we made a determination that in fact it couldn't be this script that was generating the traffic.

All of this information was relayed to the customer that we did not believe that the traffic in question was related to this script because it would not rely on UDP, an insecure protocol to deliver files to a secure endpoint where data integrity was of the utmost importance.

Unfortunately, we could not unlock the account at this time because the information we received was not clear and we already had two incidents of outbound UDP traffic that appeared to be disruptive and abusive in nature totaling 1Gbps as if it were a denial of service attack, typically associated with UDP packets.


A far more thorough response than I expected to see. It does look like, best case scenario his instance was compromised and being used for malicious purposes, worst case scenario he was actively doing something malicious himself. In either case taking the compromised/malicious instance offline is the appropriate first response in the case of an active attack underway.

I think I have to agree with a comment someone else made in the previous thread about this, with the rise of cheap VPS services we're seeing an influx of people unqualified and unprepared to run their own internet routable servers and things like this are the outcome of that. When you choose to stand up a VPS with a service like DO you take on the responsibility of keeping it secure and preventing it from engaging in malicious activity. If you fail at that task, the consequence is your servers will be shutdown for the good of the internet as a whole. If you are either unprepared or incapable of dealing with that responsibility you should be paying for a hosting service that's prepared to offer those services for you.

I say this as someone that currently has accounts with a number of VPS providers where I do take the responsibility of managing my servers seriously, as well as previously helping to administer a server that was compromised and taken offline until such a time as we had performed a full audit and verified our code on a new instance.

You should be prepared to treat a compromise of your servers the same as any other form of disaster. Treat it the same as if a flood happened and took out the facility you were hosted out of. You should have a backup plan in place so you can roll over to your backup until such time as you can fix the "broken" server, or else accept the downtime.


I think it's a bit sad that you have to go to such great lengths to protect your company's image. This was a very obvious case of customer fault. He admitted that he doesn't know if his server was hacked. The customer probably should have looked into the issue before making a inflammatory blog post.

Maybe if you guys charged more you wouldn't get these ultra-cheap customers who think they should get 24/7 support for paying $5 a month.


Being attacked or even getting compromised is customer fault ? Okay i get that. But what about closing the account instantly, not notifying the customer, accusing that customer of being a cheap liar and treating them in a bad way?


Suspending -- what was done in the first instance -- a system which is engaging in an apparent DDOS is a perfectly reasonable action.

Locking the account when the explanation given is inconsistent with the observed behavior it was supposed to explain and the system is again engaing in an apparent DDOS is also a perfectly reasonable action.

It also seems from your posts that both times you were notified of the action by DO, so "not notifying the customer" is not an issue.

I don't see any evidence you were accused of being a "cheap liar", either.

The fact that services now make it cheap and easy to set up servers doesn't mean that you have no responsibility for what the servers you set up do. If you are really running a service with 25K active users, you probably ought to be able to respond to your VPS hosts questions about unusual UDP activity with either an explanation that holds water, or an up-front admission that you don't know where it is coming from and will take action to prevent it, rather than claim it comes from a database dump script that doesn't use UDP.


Sadly, suspending a system which is the target of a ddos is the only action an ISP can take. Even if you're not ddossing anyone but merely the target you'll get suspended, assuming you're not paying enough to make it worth it for the ISP


Sounds like they gave you plenty of chance to check it out;

""" We informed the customer that it may be a good idea to check through the virtual server to see if there were any signs of a compromise just in case """

In fact from the dates listed, it looks like you had over two weeks to check for compromise. Even then, it seems like they tried to talk to you before just closing your account;

""" A second UDP pattern was detected on 2013-09-24 12:27:09 and a ticket was opened 2013-09-24 12:27:14 to request more information from the customer. Because this was already a second occurrence we had to do a more thorough follow up. Discussing the matter with the customer, he informed us that it was a mysql db dump script that was pushing data to dropbox. """


> But what about closing the account instantly, not notifying the customer, accusing that customer of being a cheap liar and treating them in a bad way?

I know this can be difficult, but it's important to understand the difference between calling someone a liar and pointing out that what is being claimed does not match reality.

DO has to take action to protect their reputation, their other customers, and whoever is the target of the potentially malicious attack. If they have empirical evidence of this attack, then they must take action. When you attempted to explain what you thought was the source of the traffic, they took the time to show (in detail) why that was not the case. This is not the same as calling you a liar. It is calling you incorrect, but these are two very different things.

You can be incorrect and not be a liar. That's a valuable lesson to learn if you intend to work in a technical field. If I were you, I'd try to take a step back. Take a couple of days off and come back to the issue. Try to understand why everyone is siding with DO on this, and resist the urge to immediately believe that everyone is against you.


According to Ben, they did notify and talk to you about the issue.


perhaps they can't tell if the customer was running the attack or not... it's kind of similar to someone buying and selling drugs in your house... it's your house you take some heat when it happens in your home... not sure it was the right thing or wrong, but i am sure they have to shut you down at least on that one instance to protect themselves as well as anyone else using their service...


You know what they stopped answering my ticket after first response. If it wasn't HN post gaining this much traction i'm pretty sure that they won't respond to me.


They responded to you 7 minutes after discovering the irregular behavior. It's up to you to figure out the cause. It's not their responsibility to tail log files.

Had you chosen a VPS like amazon, you probably wouldn't even get notified. The first time you'd notice a problem is when you get your $1000 bandwidth bill from amazon.

You're like the customer that shows up at a restaurant and complains publicly about the food and demands a refund. When you make it a public issue, the company will move mountains to help you, but you're still an asshole for doing it.


They notified me what? They closed my account first and then mailed me after? It's like killing a man first and then saying the reason why.


If the account can reasonably be considered to be abusive (whether intentionally or because it was compromised), DigitalOcean has an even greater obligation to protect their network and the other network that's being targeted. Immediately suspending the account is the correct first step.

If DigitalOcean's support wasn't clear about their reasons for suspending the account, or if you feel that you weren't getting a helpful response from them, then post the communications you had with them to prove it.


Well, we've now heard two different stories. According to Ben, there were two events. The first time they took your server offline and contacted you to tell you about it, they did not however lock your account. When the second event occurred with you being unable to provide a reasonable explanation and apparently being unable to deal with whatever compromise occurred, they took the system down again and this time also locked your account.

Your explanation of the events only mentions a single occurrence, at which time your account was locked in addition to the server being shutdown.

In either case, a server engaging in malicious activity, is normally taken offline as soon as the malicious activity is discovered to prevent further damage from occurring. You'd get a similar response from just about any other hosting provider you care to name. If you're lucky, and they're feeling generous, they might work with you to find the problem prior to taking the system offline, but normally standard procedure is to take the system offline immediately. The fact that you seem surprised about this shows you don't have much experience administering your own servers.

The standard response usually goes something like:

1) Server is discovered doing something malicious

2) Server is taken offline/shutdown

3) Administrator is notified

3a) Read only copy of the old server HD is brought online on a new server to allow administrator to perform forensic and backup work*

4) Administrator must bring up new server to replace old compromised one

*Sometimes the provider will provide you the old HD image, sometimes not, really depends on the provider.


They basically killed the killer, 1gbit udp probably killed the website or servers of someone else.


Suspending account is an action reversable with sufficient cause, killing him is not.


This is:

a.) pretty good evidence that DO did their due diligence and

b.) way more thorough investigation than I would have expected from a "value" vps provider.


So you shutdown the VM due to what appears to be a compromise. That is typical host behavior at least.

Locking the user out of their account seems...odd, tho. I've never had that happen, personally, at any host.


Maybe Ben can explain what "locking" an account is actually is. If the customer did, in fact, break the TOS, I think it's logical to lock his account.


Locking the account should put it in a read-only mode. No changes to your DNS, no ability to start the droplet back up, firewall rules put in place at the hypervisor/network layer that block outbound traffic, but still allows you inbound access.


That would be ideal. Its sounds like it was more than preventing changes to the account information tho.


Good reply Ben. I am actually a DO customer I think you've explained yourself well. Good to remember that there are always two sides to the story. I'm sure both parties could have done things differently. But thanks for the disclosure.


Instead of locking the account, couldn't you just have blocked all outgoing UDP packets from the IP address of the customer's droplet?

Blocking UDP would have stopped the attack, it would have given you a time window to contact the customer (allowing for time zone differences) and would have given both parties a chance to resolve the issue in private and with much less drama.

(speaking as a current customer of DigitalOcean, using UDP (thru collectd) to monitor my droplet and starting to have uneasy feelings).


After 1 day or so i tried to reach my server but couldn’t even ping or ssh to my server. I thought that DO was down but the truth was that they’ve locked my account without any notification. Can you imagine this?

The customer is lying in this case?


I am curious to see the blog poster's response to this, as your course of action seems completely reasonable.

May I ask what about the UDP traffic made it appear abusive?


how about 'totaling 1Gbps'?



Hello Ben, thanks for the response. Fırst of all at first ticket i told that the only possibility of having an UDP outgoing is that script that i wrote.

Other than that i've no other activity or script that can generate that much traffic. Haven't you even considered that my droplet may be compromised or being attacked ?

Instead of letting me know what exactly happened or which processes were running at that time you just locked the account and accused me.

Couldn't you even look at the access logs or so to see which IPs login into the droplet and then take your action later instead of closing it instantly?


Most cheap VPS providers would not go do that much work to decipher the root cause of the problem. When you have a server sending 1Gbps traffic, it's easier to shut it down than tail log files to figure out the cause. Digital Ocean can't do that for every customer that pays $5 a month. If you want a high level support, you should have picked a mid-level VPS like rackspace.


Looks like they looked at the code and determined it could not be UDP from your script.

I think they did consider your droplets to be compromised.

I think at issue here is that they have to assume that the droplet owner is the malicious party, if they don't lock your account, they can't stop you from creating more droplets.

I think you may have a point that they did not clearly explain to you why your account was locked.

However, this incident makes me more likely to continue using Digital Ocean. With the new private networking they have in NYC2, I for one am thrilled that they do this kind of proactive monitoring.


Unmanaged = Your responsibility

I got my server down with DO many times at first. It was a problem with the CentOs package; but it was indeed my responsibility to fix it.

You are responsible of securing your server.


According to Ben's account, they did suggest to you that the droplet might be compromised. That is what they believed to be the case.


I was also tremendously happy with DO and their service. But what if you get your production apps down without even any notification and proper reasoning ? That's the thing which makes you feel insecure.


They did notify you.

>Our monitoring picked up a malicious UDP traffic pattern on 2013-09-08 00:58:23. A ticket was then opened with the customer at : 2013-09-08 01:05:55 roughly 7 minutes later.

Also, you should do a better job securing your server. It seems like the server was compromised.


I'm sympathetic to your frustration — but when your box is owned and actively participating in illegal activity, you kind of have to expect it to get shut down.


If you have a production app, it would make sense for you implement HA, then you wouldn't have to worry about a single server getting hacked.

What proper reasoning do you need? If your server is hacked, it makes sense to shutdown the server, or disconnect it from the network completely. You can extract the data at a later time.

What if your server was hacked, then started serving up child porn? Would you be okay with having the server continue running?


The only thing who feel insecure here is your vps.


About a week ago I got to experience Digital Oceans very tight suspension policy first hand. What happened was that one of the accounts I manage was suspended. I had to go through a very long and detailed validation process before they understood that they had done wrong (they admitted to doing an error and apologized). But when my account was activated again my droplet had been destroyed..

After some more time they managed to resolve this and I'm again a happy DO user but I wish that they take a look at their policies. Just the fear of knowing that they can shut you down by mistake for a day or two is bad enough to not use them. They should have a policy where they at least call you and talk to you before they do anything.


These kinds of seemingly random suspensions at DO are starting to concern me. I've got a service with ~10K active users I'm migrating to DO as we speak, but man! I'm starting to feel like it might be professionally negligent to do so if they're trigger-happy with shutting down instances.

It'd be nice if DO had some way to communicate "hey, this VM matters to my business, PLEASE don't do anything stupid/automated without contacting me first", but that's probably too much to ask for the cost.

Gah. Back to AWS. This sucks. :/


Some months ago a tried to open an account with DO, but my account was blocked by DO before creating a droplet because my dad, who has the same name as I do, had already used the service. DO wanted me to verify my identity, address, etc.

I can't stand such friction, so I stayed with AWS, who have never had an issue with having a customer with very similar or equal name to another. Clearly, there is something wrong in DO's identity/fraud detection process, and even more wrong is the fact that they are locking user's production accounts without any warning at all.


I ran into the same problem just last week when I tried to make an account to try it out.

I added a credit card to my account without a promotion code, but later on found out about the new promotion they are doing (free $10). I realized that you cannot enter a promo code if you didn't do it the first time you add payment to an account, even if the account is otherwise still new (I didn't do anything with it after adding the payment card).

So I de-activated my account, and made another one, entered the same payment info but with the promo code this time. Then my account was instantly locked for using the same credit card, and I received an email shortly after citing section 2.6 of their ToS which prohibited users from using the same payment card on multiple accounts (even though my previous account was de-activated).

Their trial/promo code system really needs to be revamped as I spent way more time figuring out how to put the code in than actually trying out their product. Also their fraud detection system is perhaps too sensitive? I mean I should be able to re-use payment cards if my old account was de-activated...


Ya, I contacted them to get them to apply the promo since I didn't have alot of faith in their automation/tooling.

Honestly, for the price, some level of issue/limitation. If you are charging 50% [or more, depending on whom you compare to] of the price when compared to a major competitor, you have to cut somewhere. It seems DO has uneven support quality [like any low end provider] and smaller features set [combination of time and $$].

Then again, people think I'm too generous with Linode and the issues they've had [they cost twice as much for similar specs to DO] so maybe it isn't just the price point.


They have really quick customer service; they probably would have added the promotion to your account if you had contacted them.


Probably. But at the time I thought it was a problem I could've taken care of in less than a minute, basically de-activate my account then create another one and add the code. In this case I chose not to contact them, because as fast as their support can be it mostly likely won't be faster than I can de-activate and make a new account. But I was wrong as my account was locked :(

Another thing is their ToS. I admit I didn't read their ToS prior to registration (most people don't I assume, but I won't use that as an excuse), but I did go back and read it afterwards, and it said:

"Users are restricted from registering multiple accounts with the same billing details without first notifying DigitalOcean of that intent to ensure that accounts aren't automatically flagged as possibly fraudulent and without notification accounts may be treated as abuse and/or fraudulent which would lead to suspension of service."

And I thought, even if I did read this I would've reasonably assumed that since my other account has been de-activated, I'm free to make a new one with the same billing information. Unfortunately I was wrong.


Wait... did I miss something or did you send snailmail and expect a response within 10 hours?

Also I am nowhere close to being a lawyer but I'm pretty sure if they disclosed your private information on here without your written permission you could sue them probably for more than either of these 2 posts are worth against them.


I think he scanned the letter and emailed it.


Sad to say, the same thing happened to me and the support messages were unhelpful to say the least. They have several copy-and-paste answers they use far too eagerly.

And it's asinine to ask someone to update billing information when they've locked your account from accessing the billing info page. Two of us have seen that with DO. Makes them look quite sloppy.


And now after nearly 10 hours or so i still haven’t heart from DO.

There is a problem on the Internet is that people demand things NOW. Really? How long will it take for stuff to happen in real life (especially if you are dealing with government).

Some stuff does happen immediately (like registering or purchasing something), but stuff which requires human intervention is obviously slow. And it requires time.

Yes, you can have 24/7 response. But only if you hire someone (probably 3 persons) working for you round the clock which will mean paying thousands of $$ per month and not $5/month.


10 hours is unreasonably long for someone who is running an application that other users need to access. People are not going to be happy if Farmville is suddenly not remembering all the cows they milked last time they were logged in. This is about keeping customers happy down the chain.

That said, I actually really like DO and I've only had prompt, helpful responses from their customer service. I don't build apps but it is an excellent personal vps, and I still recommend them to my friends when they are looking for one.


If you're paying $5 a month, don't expect to have support turn around times under 1 day. If you're running Farmville, I would hope that you're paying more than $5 a month for your server.


I use DO for a messing around on a small web app I'm developing. Within 24 hours of having my droplet up the root password was guessed and my machine was used for some DDoS. Granted I was an idiot for not changing the password immediately but I definitely felt like DO should just use ssh key validation like AWS does right off the bat. That deters attackers from even trying brute force attacks in the first place.

Anyway, I checked the logs and pretty much the minute my machine was deployed a script was guessing my password (lots of failed login attempts for "root" and "oracle"). This probably means someone knows DO's IP addresses and their automatically generated password scheme (all lower case alpha characters of a fixed length).

I reported the incident and destroyed my droplet since there was nothing important on it. When I heard back from DO I basically got (paraphrasing here) "you should install fail2ban next time". Case closed. I'm not a big customer or anything so I don't expect premium support or anything but I feel like someone should have looked into the attack a bit more. Seems like a lot of people are experiencing the same thing.

I guess what I am saying is you get what you pay for (it is only 5 bucks after all).

EDIT: Still using DO. I was just a bit more careful next time I deployed a droplet.


> Is it fair to tyrannically close someone’s account up and accuse him or further treat him as a liar ?

Isn't this what Google does on a daily basis to users of their various services? There seem to be no consequences for them and less than zero interest in improving any of it.

There are companies that can cause your business untold financial damage through these kinds of actions. For some reason they continue to evade responsibility in financial, moral and ethical terms. It sucks.


The difference is: you are the product for google and not the customer. Trust me, google doesn't treat advertisers the same way it treats free gmail users.


Google often shuts down advertiser accounts with little or no notice. You only have to search for "adsense" or "adwords" along with "banned". Some of those are with good reason - others are not.


There may be all kinds of valid reasons to validate acounts, check documents and ownership of cards, etc.

None of those reasons excuse putting services offline while this validation is happening - first try validation, give it some reasonable timeframe, and only then cut or don't cut the service depending on results. Is it really so complicated?


OK, after read the two blog posts, and all the comments here, this is my tough:

I've a poor's men backup solution too, from my personal droplet to gmail.

I use it for backup /etc, /root, /usr/local/(s)bin and /html, with excludes and rotations (about 15MB total), encrypted 4 passes with 4 different algorithms.

I did write two scripts, one for backup-restore (and encrypt/decrypt, secure deletion, html email generation, etc) and other to list-retrieve remote backups using IMAP.

On gmail I did put a filter to send all to the trash, so I get 30 days backup rotation. More complex rotations could be implemented using different cron tasks, with different config files, pointing to different accounts (srv-month-number@...), but for my personal VPS I don't need that, 30 days is enough.

But.

Now, I go to the digital ocean panel, to the "backups" tab, and I read:

   "Pricing is set at 20% of the Droplet's monthly cost (e.g. It will cost $1/mo. to enable backups for a 512MB Droplet)."
And I can only think: facepalm.

It's not only the price of the implementation time, that me and this other person with ruby/dropbox have spend...

Even if it looks like "you get a gratis backup solution", really it may be more expensive (because of all the network bandwitch)

Well, at a side of the fun of implementing your own backup/restore scripts, you get:

* provider independence (if all DO is down, you still can restore from gmail or dropbox)

* no periodical costs (but remember the implementation cost maybe bigger the first time, then is just reuse and edit a few variables).

* Security (you control how and where the data goes, otherwise you can save and restore from the provider, but implementation, management, internal policy, budget, team or technology changes, etc around your data is up to you).

But, I will never call again my solution a "poor's man" solution, because counting implementation time and network traffic, is much more expensive than $12/YEAR.

Edit: formatting a list.


Previous post was submitted here: https://news.ycombinator.com/item?id=6438761


I left them long ago after super crap support.

No thanks and good luck.


Most online services that are under $5 a month rarely come with support. I feel bad for startups that charge so little per month because usually the less you charge, the more demanding the customer is. Virtualized servers tend to require more support than say, online accounting or email products.

People who go to Digital Ocean are going for cheap VPS solutions. If they had money to blow, they'd probably be with amazon or rackspace.


> And now after nearly 10 hours or so i still haven’t heart from DO. Also now that the HN topic is not on the front pages no one is getting updated about the situation. That’s why i wanted to write this post.

I won't speak for everyone, but I can personally live without having updates on the OP's Digital Ocean drama on my HN homepage at all times.


> DigitalOcean requested me to send a paper which authorizes them to disclose the information in public. It was midnight here in Turkey and i was asleep.

> And now after nearly 10 hours or so i still haven’t heart from DO

What about timezone differences? You were asleep, they could be, too.


That's a possibility. But at this time yesterday they were really really active.


that's rough

they obviously only care about conversion and not validating the user before hand in (any) way, so brutal.

From verizon I got a similar answer they couldn't discuss my own account with me after me giving them my social and all that - wtf. i hung up and tried another service rep and they gave me too much of my own personal info with out verifying any of my personal details and anyone elses' on my plans, which is kind of scary (Wtf)

also i really really like the idea of daily backups to dropbox, with like 1TB of backups i need a solution, and this seems like a great one. thanks for coming up with that


you're going to back up 1TB of data to dropbox?


Funny that DO's public response was to demand permission to discuss the case publicly, when apparently they had not discussed it with the customer concerned!


I think they have. The customer in question simply didn't put it in his blog post. I feel as though the customer may be presenting the facts in such a way to make DO look bad. I don't see anywhere in his blog post about his DDOS attack. His server was shutdown because it was running a DDOS attack. My guess is that his server got owned...

People get disproportionally mad when their server gets shutdown, for a service that charges $5 a month for a VPS. Most people probably wouldn't get angry at Netflix if they cancelled their $7 a month account, but shutting down a VPS feels like a personal attack.

If you pay $5 a month for a service, don't expect a customer representative to hold your hand and debug your server troubles.


I think that is the appropriate public response. Not discussing it with the customer is obviously an inappropriate private response.


I've also had a bad experience with DO in production. Moved to Webfaction and have never looked back. They are just excellent in every which way.


By the way, someone should watch this thread and start saving all posts by the OP. Last thread he deleted a lot of them.


I'm a DO user, I created my account 2 months ago and I'm a happy customer.


Isn't that a major violation of their SLA? Have you consulted with a lawyer?


Even if they violated their SLA, the most DO could do is offer a $5 refund, which isn't that big a deal. It's probably too costly to hire a lawyer over a $5 server.

Also, I don't think the SLA would include things like getting hacked, or running illegal services.


What do you mean by a major violation of their SLA?


Never heard on DO but will use them for a project soon - looks great


Indeed, never though a $5 provider would go that far for a noob on their network. Looks like their monitoring system is good too, took them only a couple of minute to take him down, which is good.


is this... a joke?




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: