
Digital Ocean NYC2 Droplets Down - heavymark
And counting. Second time in about a week and during prime time hours. If it doesn&#x27;t get up quickly or if happens again to NYC2 in the next few weeks may be time to reconsider DO for hosting production sites. I love them, and glitches happen, but twice in a row clients have a hard time swallowing.
======
hadoukenio
Why do people have their eggs in one basket? More specifically, whenever
hosting comes up on HN, it's always DO vs Linode. Why not Zoidberg^Wboth?

In the age of DevOps, it's not hard to configure _different providers_. I have
been using Linode _and_ DO for a while now. I noticed DO go down last week,
checked my site, and it was UP because it was being served via Linode.

I flinch every time I hear someone say to me that their service has redundancy
because they use multiple Amazon regions. Multiple providers people! And
preferably multiple providers in multiple states, paid for by mutually
exclusive credit cards!

Redundancy doesn't mean having multiple hosts by the same provider, within the
same data center, within the same rack, provided by the same power strip.

tl;dr: Plan for outages.

~~~
anderspetersson
How do you do this on the DNS layer? Very short TTL and switching on failure
detection? Do you use both providers at the same time when both are up via DNS
round robin or similar?

~~~
hadoukenio
There's nothing stopping you from adding multiple A records for a domain. In
fact, that's what a lot of people do:

    
    
      dig a www.google.com
    
      www.google.com.         300     IN      A       74.125.237.178
      www.google.com.         300     IN      A       74.125.237.177
      www.google.com.         300     IN      A       74.125.237.176
      www.google.com.         300     IN      A       74.125.237.179
      www.google.com.         300     IN      A       74.125.237.180
    

Browsers these days will resolve and try each one, on a first-come-first-
served basis. If one fails, it will try the request on the next server.

------
bigredtech
I'm a big fan of Linode. Have had one of their mid-range servers for about 2
years now and have had no troubles. When it came time to upgrading, or any
other questions I had, their support staff is 1) quick, 2) knowledgable, 3)
thorough & helpful.

I've used DO in the past for quick spin-ups, and they're great for that. But
for long running applications I'd stay away. Was actually considering DO for a
new project, but this kind of tipped the scales the other way.

If you're looking to switch, I think Linode gives you the best bang and
reliability for your buck. Have also used Rackspace for a previous startup. A
bit more pricey, but also a great service if you're considering switching.

~~~
meritt
Linode can't silo resources on a per VPS basis and constantly will harass you
for over-utilizing your share of arbitrarily enforced limits. Just a heads up
if you actually utilize your instances.

~~~
harpastum
Are you talking about the "Linode Alert" emails? Those are user-configurable —
in the email, they specifically state _" This is not meant as a warning or a
representation that you are misusing your resources. We encourage you to
modify the thresholds based on your own individual needs."_

If that's not what you're talking about, I apologize, but I've been using
Linode for years (often with very compute-intensive projects) and I haven't
seen what you're talking about.

~~~
jarin
Just out of pure laziness, I use the CPU and disk access emails as makeshift
notifications for when my backups run.

------
kl4m
Their status page indicates "We have identified a slow connection to our NYC2
data center". Assuming they have redundant connections, this seems like a big
understatement for being down for 1 hour and going. Even their control panel
is down for me. (
[http://i.imgur.com/3ltg00o.png](http://i.imgur.com/3ltg00o.png) )

------
neom
we've had an ongoing issue with a juniper fiber core in nyc2 it seems. We've
engaged juniper to help us, I don't expect this to be an issue again.

------
jcampbell1
Linode's new offering probably the better choice these days. I get the feeling
that DO is just overwhelmed with problems related to their hyper growth - 10%
per month on a huge base.

[http://trends.netcraft.com/www.digitalocean.com](http://trends.netcraft.com/www.digitalocean.com)

~~~
mctDarren
They've done so well to this point keeping up with the growth curve. Hope they
don't become crippled by success.

------
jqueryin
Having previously left Linode for Digital Ocean due to lower pricing and SSDs,
I can safely say Linode both dropped their prices and matched the DO plans
with SSDs included. If you want to jump ship, that's the route I'd go for a
comparable.

I still love DO, but downtime is a no-no when it comes to VPSes.

~~~
JohnTHaller
When you charge so little and the SLA is just credit during the time of the
outtage (pennies), downtime is pretty much a given. It always surprises me
that this surprises people.

------
JohnTHaller
You should probably reconsider hosting production sites on super-cheap VPSes
with a 1:1 SLA refund. They're going to go down. And, when they do, you'll get
the 7/10ths of a cent refund per hour promised for your $5 instance (1.4 cents
per hour for $10, 2.8 cents per hour for $20, etc). But if you depend on a
site for your business, ask yourself how much it costs you per hour of
downtime and plan appropriately.

For context: I'm a happy Digital Ocean customer (as well as Rackspace and
others). I like their services and consider them a great value for the money.
I've also gotten multiple developer friends into using them. But I don't let
them be a single point of failure for anything critical.

~~~
kbar13
downtime is an issue no matter how much you pay your infrastructure provider.
You should account for downtime in your application and your infra by taking
advantage of low-hanging high availability fruit.

Things like saltstack (saltstack.org) make this mega-easy.

~~~
JohnTHaller
It's easier for apps since you can build in failover support based on multiple
servers pretty easily (I just implemented this in our 'live' installers
recently on Windows). It's a bit harder at the website level for an average
user, though. I know far too many people that have a single point of failure
website on a super-cheap VPS that's just down whenever the VPS is down. Most
of them would have no idea how to implement a load balancer and failover
support. Though I could probably get them into something like CloudFare or
similar.

------
Sir_Cmpwn
I am getting pretty sick of it, too. I might go back to Linode. I was already
upset that they dropped Arch Linux support.

~~~
opendais
Linode has price matched them so its not a price issue anymore. [e.g. $10
Linode is roughly the same as $10 DO]

Glad to see the ProjectWonderful ads providing something, even if its small
btw. :)

~~~
Sir_Cmpwn
It definitely helps. The biggest source of income has been (surprisingly)
Gittip, though.

~~~
opendais
Ya, I am surprised by that too. I'd create one but I'd feel guilty. My ads
cover my costs.

------
wglb
Er, no.

My droplets are up. Never went down today.

Also, please don't post that things are down here on HN. This is not a support
forum. Not for DO, not for AWS.

~~~
iurisilvio
You came here after it was fixed. It is fine now. Also, it was just NYC2,
maybe only part of it.

~~~
wglb
Mine are on NYC2. Terminal sessions up all day. On three of them.

Check [http://downforeveryoneorjustme.com](http://downforeveryoneorjustme.com)
instead of HN.

~~~
anatoli
Or you could check the official status page[1] and see that there was indeed
an outage.

[1] [https://status.digitalocean.com/](https://status.digitalocean.com/)

~~~
wglb
Even further reason not to post it here!

------
uptown
Here's their status panel:
[https://status.digitalocean.com/](https://status.digitalocean.com/)

------
potomushto
[https://cloud.digitalocean.com/](https://cloud.digitalocean.com/) now is
unavailable too.

------
larrys
"but twice in a row clients have a hard time swallowing."

May I ask what your clients are paying?

Maybe the answer is to charge them more instead of giving them a low price (an
assumption on my part of course which is why I ask what you are getting for
this) and some implicit guarantee of uptime.

For example we had some customers some time ago who were pooled on a shared IP
address for sending SMTP service. Occasionally they would experience problems
with mail reputation on that shared IP. If they complained about that would
simply explain that that was the drawback of a shared IP and that they could
always upgrade to a service level where they got a unique IP and could
establish their own reputation.

------
opendais
Folks, I strongly, strongly suggest if you use Cloud/VPS providers to look
into something like Cassandra, Galera, Mongo, etc. that can do WAN replication
and using 3 DCs.

Yes it costs money _but_ things like this are avoidable. Every single
Datacenter on the planet has downtime and/or performance issues from time to
time. Hell, one of the places I have a dedi at had its network performance
drop by 50% for about 15 minutes while they did emergency maintenance.

Things like this are going to happen as long as you depend on one provider.

------
Donzo
I've had great experiences with them. I recommended DO to everyone.

Now I'm tempted to drive to New York and punch them each in the face.

PEAK downtime.

EDIT: $160 refund. Maybe just a Charlie horse.

------
publicfig
I just created a droplet today, and was trying for the life of me to figure
out what I would have done so early on in setting it up to cause it to become
unresponsive. Glad to see it's not just me. It's not really a great sign
though, as I was hoping to use this fairly frequently. Is this a common
occurrence, or do I just have bad luck? If so, I'll definitely have to find
another solution.

~~~
zhodge
I'm not too qualified to comment on the hosting market, but I do want to vouch
for DigitalOcean as I've never had uptime issues with them until now. They
also have always had timely and helpful support and their pricing is obviously
competitive.

That being said, near 100% uptime is really desired in this space and I've
been scratching my head for the past hour or so wondering why I couldn't
connect to my mongo process running on my droplet...maybe I should have tested
DO sooner, but the fact that only mongo seemed to be failing (and not postgres
for example) made me think I had client and not server issues.

~~~
gingerlime
I've been using Linode for many years, and DO for some time now (a year? not
sure, since I can't even login to their control panel right now and check).
I've seen issues on both providers, and dealt with support of both not too
infrequently.

I'd still say Linode is miles ahead in terms of expertise and responsiveness.
Most of my servers are on Linode for this reason, and only a few on DO.

As a customer I enjoy this competition and would like to keep both players
compete - because I'm benefiting from it. So I'm likely to stick to both. But
I would still recommend Linode more easily than I would DO.

~~~
zhodge
Thanks for your input. I might be switching over as a result of the recent
downtime and the support Linode has been receiving on this thread.
Unfortunately, especially for non-hobbyist apps and services, this is an area
demanding fickleness.

------
mbreedlove
I'd love to know what's going on. My website hosted on DO,
michaelbreedlove.net, is unavailable. I also have a few services running on
subdomains of the same DO VPS instance; they're all accessible. I also have
SSH access. They seem to be having some serious HTTP routing issues.

------
alokyadav15
Yeah , NYC2 seems affected
[https://twitter.com/digitalocean/status/494170340973887488](https://twitter.com/digitalocean/status/494170340973887488)

------
marcusr
It seems to be http (perhaps load balancers) rather than the instances being
down - I'm ssh-ed into a droplet and everything is working fine, except remote
HTTP connections.

~~~
publicfig
I'm struggling to ssh into droplets, but it seemed like when I was ssh'd in, I
didn't have an issue at all.

~~~
mikeash
It seems to be intermittently up and down. So your connections all die, and
you can't reconnect for a while, then it comes back and everything looks good
for a bit until it dies again.

~~~
heavymark
Same here. Since 1:24p, Pingdom has reported our sites going up and down about
16 times.

------
rrss1122
We're getting the same thing. I don't care how cheap it is or how nice the
customer service is, but if they are not up 99% of the time they will not be
my web host.

------
JohnTHaller
I'm back up as of 2:59pm EDT, so I was down for about 1:35. This is the only
outtage longer than a few minutes I've had with DO thusfar.

------
dangoldin
One of my instances on NY2 is fine and the other one is sporadically
accessible via SSH. Hope they figure this one out soon.

------
bmoresbest55
This is a little ridiculous. I hope not to have this problem again. Second
strike her. Unfortunately.

------
alokyadav15
its working now , atleast for me .

------
n0nenet
I have to agree with you on this one.

