

Cloudflare DNS problem - sinak
http://support.cloudflare.com/

======
packetized
Dyn also appears to be having a real, real bad day. Glue records vanished like
a fart in the wind.

    
    
      $ dig @8.8.8.8 ns2.p29.dynect.net
    
      ; <<>> DiG 9.7.3-P3-RedHat-9.7.3-8.P3.el6 <<>> @8.8.8.8 ns2.p29.dynect.net
      ; (1 server found)
      ;; global options: +cmd
      ;; Got answer:
      ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 2690
      ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
    
      ;; QUESTION SECTION:
      ;ns2.p29.dynect.net.		IN	A
    
      ;; AUTHORITY SECTION:
      dynect.net.		600	IN	SOA	ns0.dynamicnetworkservices.net. hostmaster.dyndns.com. 2015070600 10800 1800 604800 1800
    
      ;; Query time: 2 msec
      ;; SERVER: 8.8.8.8#53(8.8.8.8)
      ;; WHEN: Mon Jul  6 21:42:48 2015
      ;; MSG SIZE  rcvd: 120

~~~
mattbeckman
Does CloudFlare use Dyn or are these unrelated issues?

~~~
rdl
The affected zones are those cnamed from CloudFlare to Dyn.

~~~
sajal83
Any example hostname thats configured on Dyn and is not resolving?

------
rdl
Sorry about this: just posted a blog post. Appears to be a Dyn issue, but
affects a lot of our customers (including support.cloudflare.com which is
Zendesk)

[https://blog.cloudflare.com/the-internet-is-a-cooperative-
sy...](https://blog.cloudflare.com/the-internet-is-a-cooperative-system-dns-
outage-of-6-july-2015/)

------
jgrahamc
This appears to not be a problem with CloudFlare, but a problem with Dyn:
[https://www.dynstatus.com/](https://www.dynstatus.com/). Of course, Dyn and
CloudFlare have mutual customers and so if a Dyn has a problem it's possible
for a mutual customer to have a problem.

~~~
stockliasteroid
Strange, we are using CNAME apex resolution with cloudflare to point to Heroku
apps, so I'm not aware that Dyn is used anywhere in that chain, but resolution
is still down for those apps.

edit: resolved now.

~~~
jgrahamc

        $ dig +short NS heroku.com    
        ns1.p19.dynect.net.
        ns4.p19.dynect.net.
        ns3.p19.dynect.net.
        ns2.p19.dynect.net.

~~~
stockliasteroid
Thanks!

------
sinak
More details here on Cloudflare's status page here:
[http://www.cloudflarestatus.com/incidents/m8hf8q8vs81d](http://www.cloudflarestatus.com/incidents/m8hf8q8vs81d)

------
rdl
Sorry -- our ops and engineering teams are looking into this and will
update/resolve as quickly as possible.

[https://www.cloudflarestatus.com/incidents/m8hf8q8vs81d](https://www.cloudflarestatus.com/incidents/m8hf8q8vs81d)

~~~
iaw
Good luck, this is really rough. I encountered a 101 on a random site before I
saw this and was a bit baffled.

------
kelseydh
I love how even
[https://support.cloudflare.com/](https://support.cloudflare.com/) is down by
this.

~~~
colinbartlett
They do at least use a different DNS provider for their status page:
[http://www.cloudflarestatus.com](http://www.cloudflarestatus.com)

~~~
Someone1234
I do love how everything is operational...

I see the big blurb about DNS at the top. But it still feels like the table is
misleading. When they're having a major DNS outage and every site is
"operational" it feels misleading.

Although if this was Amazon's AWS console we wouldn't even have the blurb at
the top...

~~~
colinbartlett
It's frustrating for me, too. Because my side project
[https://statusgator.io](https://statusgator.io) pulls in this status data and
sends alerts to subscribers when services go down.

But since CloudFlare has not indicated any downtime, everything still appears
up.

------
kelseydh
If you're on Heroku, one immediate way around this problem is to guide your
users towards your heroku domain. I've been doing this for clients contacting
me who absolutely need access right this moment.

E.g. [https://myappname.herokuapp.com](https://myappname.herokuapp.com)

~~~
Someone1234
That's a nice tip.

Can I ask a question while we wait for Cloudflare to fix this... Heroku looks
nice. But also quite expensive after you add on dyno, Postgre/Redis, and
"addons." What about Heroku do you like/appreciate that justifies the slightly
higher costs? Or do you just like that all of the devops-style stuff is
handled by them (as opposed to AWS/Azure/Linode, etc where you'd have to do it
yourself).

I am genuinely asking, I have no reason to dislike Heroku, and I know it is
quite popular. Just trying to understand what the "value add" is over other
cloud services (or why they're dissimilar).

Please don't feel obliged to answer, but if you have the time it would be
appreciated/interesting.

~~~
kelseydh
No problem, here's my two (relatively unorganized) two cents:

With Heroku (frankly also thanks to Ruby on Rails) I can be a sys-admin,
developer, database administrator and QA all at once; far easier than before.
For the price of $100k I have seen developers build successful products that
used to cost $1M+ to develop.

On Heroku you definitely pay for convenience and ease of use, knowing that you
get a benefit in doing so by being able to focus on the development of your
app, the part that matters. The costs grow if you get traffic, but through
careful management (such as packing multiple processes into one dyno), you can
reduce costs. Even if many find Heroku's prices high, they are still much
lower than what you see in enterprise land, or even on AWS if you aren't
careful.

Heroku's documentation is also incredibly good, and this helps you a lot to
move fast. Almost by accident, simply by following Heroku documentation I have
found that developers end up following good cloud server practices almost by
default (i.e. because Heroku forces you to build your app to follow certain
conventions, conventions needed for making your app horizontally scaleable;
such as delegating clock processes to a separate process, and not allowing
dynos to manipulate or rely on unique file memory specific only to them).

If I architectured or configured my own servers it would have taken 10x as
long, exposed me to far more security issues, and I would have probably
screwed it up any way. The lesson from this is that I think as developers we
need far less freedom to configure than we think we need.

So in conclusion, if you're a small to medium-sized app that does not have
complex server or bandwidth needs (i.e. you are not Imgur or Pied Piper), then
I highly recommend it. I'm sure many Oracle salesman will be unhappy with this
opinion, but take my word for it that the vast majority of applications do not
need complex, overengineered server systems to be successful. (After all as a
startup, you should not be wasting months planning on how you will scale to
support 10 million concurrent users. I've seen that hubris distract failing
startups before.)

~~~
toomuchtodo
> With Heroku (frankly also thanks to Ruby on Rails) I can be a sys-admin,
> developer, database administrator and QA all at once; far easier than
> before. For the price of $100k I have seen developers build successful
> products that used to cost $1M+ to develop.

Unfortunately, Heroku is down _constantly_. Don't take my word for it; google
"Heroku down" and see how often it occurs.

Microsites? Sure. Hobby sites? Sure. Business applications that generate
revenue? Good luck.

~~~
kelseydh
Heroku is just a layer on top of AWS. I know a _ton_ of successful business
applications who rely on Heroku + AWS. Obviously after a certain size you will
want to customize your hardware more, but in the early stages Heroku can take
you a very far way.

Ello is a recent app that had to scale to millions of users extremely quickly
and did on Heroku. Read their technical write up:
[http://mikepackdev.com/blog_posts/40-5-early-lessons-from-
ra...](http://mikepackdev.com/blog_posts/40-5-early-lessons-from-rapid-high-
availability-scaling-with-rails)

Ello had issues, and Heroku would be considered amateur for what they were
doing, but if you read it Heroku was not the cause of their performance issues
-- their application code was. IMO most business applications run into
problems due to performance issues within their application code, not because
of server downtime.

Regarding how common downtime on Heroku is... yes, Heroku has lots of minor
incidents impairing its toolkit and some services, but these are not issues
that go notices. Two years in I still haven't had significant downtime from
Heroku itself (more than 5 minutes) during business hours where it matters.
Cloudflare, and other DNS providers, have frankly given me more trouble.

------
kelseydh
Lately my number one bloody cause of downtime has been DNS lookup failures. I
switched to Cloudflare hoping to avoid this. If I can't trust Cloudflare, then
who the heck can I trust?

Also, is there any way to actually resolve this issue quickly while Cloudflare
deals with their fuckup? I don't like being at the mercy of their support team
while my site has literal downtime.

~~~
elmin
I've never had any trouble with Amazon Route53.

~~~
zrail
That's because Route53 is basically bullet proof, such that their SLA
guarantees 100% availability. I have switched everything I touch to Route53
and have had zero issues.

