

DNSMadeEasy under major (over 50Gbps) DDoS Attack out of China - KrisJordan
http://twitter.com/DNSMadeEasy

======
makmanalp
I'm interning at a major managed DNS provider and this actually happens more
often than you think. Most of the time it ends by working with telcos like
level3, cogent etc, blackholing routes, sacrificing a certain geographical
location in favor of keeping others alive and basically just waiting it out.
It's interesting how because of the nature of the internet, DDoSes are hard to
deal with.

~~~
drivingmenuts
Too bad we can't just permanently sacrifice China.

~~~
dnsworks
There's an anti-DDOS service out there, Progent or something. Part of their
solution adds all of APNIC's ip addresses into your router's bogon list when
you're under attack. It turns out for most DDOS's if you turn off china, the
attack just ends.

~~~
tptacek
I think you're thinking of Prolexic, and my recommendation is that you take
any of their claims with a heaping helping of salt. Things may have changed in
the 5 years since I've totally forgotten everything I knew about DoS
mitigation, but it wasn't the case then that you could simply black hole China
to evade them.

A more common mechanism used by real networks to mitigate these attacks in
production is to pinpoint the _target_ of the attack and offramp its traffic
in the ISP core to a "regional scrubbing center" where advanced filtering
tools (for instance, packet filters that can handle ACLs with high tens of
thousands of terms) can try to sort through the crap.

~~~
jacquesm
At my colo that is exactly the sort of strategy they use, they have bought a
bunch of routers that use FPGA based filters for that.

The one scenario I can think of where that might be a problem is if they'd
start flooding all the known hosts in a network for the specific purpose of
overwhelming the routers. And even those hardware based filtering tools have
upper limits.

~~~
dnsworks
There's really no need to do that. A large enough BotNet can take out almost
any host doing nothing more than connecting and doing a GET / HTTP/1.1. Few
websites can handle a sudden surge in traffic from 100,000 bots.

~~~
jacquesm
The idea behind an ACL for such an attack is that the same hosts will be used
over and over again, taken from a large pool of zombies. So, let's take your
example of 100,000 bots, an ACL in the upstream router (as seen from the host)
could be used to checked against to identify those packets from the zombies
and discard them. After all, a single attack of all 100,000 bots at once will
just bring the host to its knees for the time-out of the connections and then
it will bounce back up again. So, to increase the effectiveness of the attack
they reconnect after every lost connection asking for another resource. If
they're smart they'll vary agent strings and other characteristics to make it
hard to narrow down who is and who is not legit.

Initially you don't have much to go on during such an attack and packet
filtering is a reasonably expensive operation when you want to do it for a
large number of hosts. So the strategy is to route all the traffic destined
for that particular host through a router that has ACLs that are large enough
to hold the total IP list for the botnet that is attacking the host, as these
IPs become identified.

You don't want to route _all_ your traffic through there because then you'd
have to do the relatively expensive filtering on all of the packets, even
those not destined for that particular host.

Now if an attacker were targeting the hosting facility they could thwart this
strategy by sending requests to a larger number of hosts in the network in
order to make life much harder for the crew fighting the attack. After all,
you can't partition the problem anymore in to a portion that is targeted to
the host and 'normal' traffic, effectively all the traffic could be bot
traffic or it could be normal traffic, for all the receiving hosts.

To be able to partition the problem into a smaller one where you can let say
90% or more of the traffic through unfiltered and only concentrate on the
remaining 10% would make solving it a bit easier. On the other hand if the
attackers are silly enough to re-use the same bots to attack different hosts
they've actually given you a clue as to which IPs are bots.

I hope that makes sense :)

------
FooBarWidget
I'm wondering what the implications of DDOS are for website owners. What if
you're on EC2 for example, will you be charged for the 300 TB of traffic? If
so that would be an easy way to bankrupt a startup.

~~~
dennisgorelik
The implications for website owners are that their web sites were temporary
unavailable. Traffic simply does not reach the web site. Even normal traffic.
For example, you type in your browser www.mywebsite.com. DNSMadeEasy would
normally resolve it to your IP address (e.g. 111.222.123.12). But because of
the DDoS attach -- mywebsite.com cannot be resolved into any IP address and
you cannot open www.mywebsite.com at all.

~~~
dan_manges
That's the implication is there is a DDoS attack on your DNS provider, but I
think FooBarWidget was inquiring about the implications of a DDoS attack on
your website, and specifically if you would be charged for the bandwidth
consumed by the attack.

~~~
jacquesm
You can - but not everywhere - negotiate that the service you pay for includes
protection against DDOS attacks and that it's up to your provider to protect
you. You'll pay a larger fee per mbit because they'll need to do more work for
you in case you get hit but it might be worth it.

------
leftnode
No wonder most of my sites were down or not functioning very well this
morning. Hope they get everything operational soon, DNSMadeEasy is a great
provider.

------
quizbiz
Can someone please explain to me what DNSMadeEasy actually does?

Is a domain provider like 1and1 a client of theirs or is my hosting provider a
client?

This entry on Hacker News led me to wikipedia where I went from an article
about Level3 to an article about Tier 1 Network to Internet Backbone. I feel
like I'm on the verge of understanding more about how the internet works but
it's all a bit above my head.

~~~
geocar
They're an anycast dns provider- basically the only kind of third-party DNS
service that's worth anything at all. They have some neat domains-related
tools that hook into their infrastructure that's kindof a pain to do yourself
(even if you've got your own anycast-capable network)

------
wehriam
What motivates an attack like this?

~~~
Steve0
Probably some motive against one of the customers. If you can extort some cash
out of a gambling site with short dns-ttl it could be worth your effort.

Maybe the target's data center is more ddos-proof than the one from easydns,
extortionists go for the weakest link.

~~~
mrtron
Back in the olden days of IRC, there was times when entire regions of the
internet were taken offline resulting from a ddos for personal vendettas. For
exactly that reason too - they attack the weakest link and often it isn't the
host directly.

In this case - it really could be anything. The cost of one of these attacks
is next to zero. Rarely will the botnet owner lose any machines resulting from
an attack. The unfortunate thing after one of these attacks is you have no way
of preventing it or going after the source.

------
cliffchang
Can anyone give an idea of how large, compared to other DDOS attacks, 50 GBPS
is?

~~~
lsc
it's fairly large. There are really two factors in a DoS, though, total
throughput and packet size. Obviously, your incoming pipe can only handle a
certain total throughput, I mean, that's what most of us get billed on.

However, most routers and firewalls also have a limit on packets per second
they can process, on top of the throughput limits. I've got a 100Mbps commit
on a 1000mbps pipe, and I can handle 1000Mbps of 'normal' traffic... but I got
taken out a month back by a 200Mbps DDos that used very small packets. My
router couldn't handle it. (now, if I had spent money on a better router, it
wouldn't be a problem. As far as I can tell, even, a reasonable software
router could have handled it.)

Another way to measure this is the capacity required to absorb the attack. You
can get he.net bandwidth for around a thousand dollars a month per gigabit,
and he.net is about as cheap as bandwidth gets, so to soak a 50 gigabit
attack, you'd have to have fifty thousand dollars a month of spare capacity.

(I'm sure there are further discounts available between the 1GiB and the 50GiB
tier... but you get the idea. )

~~~
ttimrawi
Considering the fact I've seen this attack first hand. I can tell you a couple
of things about it's strength. It's very flexible one minute they are sending
packet size 1500 bytes udp, the other they are sending 48 bytes syn tcp 80.
However, filtering them with a Firewall is not hard at all since they do have
packet patterns you can detect, but even if you can find a firewall and have
it on the edge of your network traffic is STILL reaching your network and if
you can handle 50Gbps of traffic all coming from a couple of different ASn
than "wow".

------
ximeng
(Brief) discussion on webhostingtalk:

[http://www.webhostingtalk.com/showthread.php?t=970837&hi...](http://www.webhostingtalk.com/showthread.php?t=970837&highlight=china)

Another 30 gbps attack, discussion on webhostingtalk.com:

[http://www.webhostingtalk.com/showthread.php?t=966658&hi...](http://www.webhostingtalk.com/showthread.php?t=966658&highlight=china)

Edit: interesting thread, apparently the reason it's 30gbps is that it's
maxing out the link from China telecom, so legitimate Chinese customers are
getting traffic dropped. FBI are involved. Suggested solution by some is to
get traffic routed over more intercontinental links possibly via peering
agreement with China Telecom, and beyond that political pressure.

------
petercooper
I wonder if, long term, DDOS attacks will be to net neutrality as spam was to
nice-and-easy e-mail setups. Now ISPs are running spam blacklists, blocking
entire other ISPs at times, and it's a nightmare to run your own SMTP server
and deliver mail reliably without jumping through hoops. If DDOS attacks
become more and more annoying, they could be used as an excuse to violate net
neutrality.

~~~
SoftwareMaven
I'm not clear on how that would work. A good DDoS attack is nearly
indistinguishable from normal traffic. How would net neutrality play into it?

~~~
petercooper
Really good spam e-mail is nearly indistinguishable from normal e-mail. Crazy
layers of whitelisting/blacklisting/new DNS settings/laws/software and
policies have been piled on top of the mail system to reduce the nefarious
effects of spam sent en masse.

While spam and DDOS attacks aren't directly comparable in terms of what they
_are_ , I'm speculating in terms of what negative effects continued and
escalating DDOS attacks could have again in terms of laws, policies,
white/blacklisting of entire networks/countries, and so forth.

------
meric
I wonder if they're going to change the text on their website: "A DNS service
with a 99.9999% uptime history is just the start! DNS Made Easy is so
confident of it's uptime record that we offer the best service level agreement
in the industry. That is why all businesses that require stable DNS decide to
use DNS Made Easy. We have an industry leading 100% uptime gaurantee and will
credit all accounts 500% of the downtime."

Maybe its a rival DNS provider behind the attack?

------
est
The GFW of China has a known vulnerability to amplify a UDP traffic to 2x to
3x as much.

