Even if the crusade to end useful UDP protocols is successful, all of that short-sighted behavior is for nothing because untraceable DDoS attacks are still a problem. Someone with a spoofing capable, 100,000 node botnet (or the bitcoin required to rent on) with or without reflection capabilities is going to be able to saturate 40gb links.
Name and shame ISPs that allow spoofing, stop wasting developers' time with this crap.
The only way to stop IP spoofing is to shame misconfigured networks. For that you need attribution.
this presentation was prepared before the Krebs On Security DDoS attacks which had a different profile and did not use Spoofed IP addresses. Krebs's DDoS attacks launched by internet-of-things devices were unusually large. Most of the non-spoofed attacks were way smaller. This does not invalidate this talk. Non-spoofed attacks are easier to block - since the source addresses are known. It is also possible to fight and eventually destroy the botnets. This is much harder when the IP's are spoofed.
Blacklists are only useful to stop only a few DDoS attacks mainly amplified ones like the ones that abuse open DNS resolvers which should be cleaned up on their own, in which case you want to be blacklisting the resolvers not the machines that use IP spoofing since they do not attack you directly.
Many (heck most) DDoS attacks do not use IP spoofing, the recent and largest recorded in history just used 145,000 IP enabled cameras, and this isn't even remotely the largest botnet out there, blacklisting an entire botnet is effectively DDoSing yourself since many of those machines can be on a network with 100's or 1000's of other computers behind the same IP which many of them would be legitimate users of your service.
Notifying ISP's also doesn't help, some ISP's can do some preventative measures but what often they do is simply kill routes to your service, again it would work if you are dealing with some ISP in Slovakia if you aren't serving any customers there (granted they actually respond to you in due time) but if you are dealing with large ISP's there is little they could do and cutting them off isn't an option.
And as for as cleaning and isolating these machines this is laughable, you aren't going to be tracking down customers, even the ISP won't do that for you, and even if it did what would it do send and email that would likely be ignored? Even if you identify customer the IP belongs to it doesn't tell you anything about the device, not to mention that the customer would likely have no capability of cleaning up their machines to begin with.
IP spoofing is a problem because there are currently ways to abuse IP spoofing to amplify the attacks, if you aren't amplifying your attack using UDP protocols you will usually not going to be using IP spoofing in the first place, and automatically banning IP addresses might sound like a good tactic until you are hit with DDoS from a big botnet within your market region at which point you can't use that any more, you might ban a few IP's that seem to have access to excessive bandwidth but even that might be problematic since it can affect legitimate users.
Only because spoofing prevents them from being reliable in any way.
>in which case you want to be blacklisting the resolvers
This approach is idiotic and its the reason these attacks are still a problem. EVERY DNS SERVER can be used if it resolves anything, not just if it's an open resolver. So your blacklist would have to include the root name servers, the .com names servers, etc.
>if you aren't amplifying your attack using UDP protocols you will usually not going to be using IP spoofing in the first place
Completely false. SYN floods and non amplified UDP floods very frequently spoof because it makes pruning attacks upstream by source impossible.
I'm not sure where you got this information, but stop spreading it. It's protecting incompetent network operators, harassing service operators, and doing little to improve the security of the Internet.
If it didn't, we're one step closer to fixing it.
Currently, DDoS doesn't require compromised hardware, because of spoofing.
But if there were no spoofing, compromised hardware would be a requirement, and we're one step closer.
1. Victim blaming is when you excuse someone's bad behaviour with the justification that the person(s) negatively affected by it could have protected/tried to protect themselves against it. That doesn't have anything to do with the question whether you should try to protect yourself, or whether you might have some obligation to try and protect others.
2. Based on your logic, what you are doing would be victim blaming: Incompetent ISPs don't perform DDoS attacks, and yet you seem to see some sort of obligation on their part to implement BCP 38 to protect others from that bad behavior of others. The ISP is just as much a victim in a position where they can help and protect others as people running UDP services.
They are also victims from the second order effects of idiots in prominent places claiming that their services are the source of the Internet's woes (e.g. Cloudflare's ridiculing of dns resolver operators).
Your entire second point is nonsensical because they aren't impacted by DoS attacks. They aren't 'victims' in any sense.
Also, people running UDP services cannot stop source spoofed DDoS attacks. You can block the entire UDP protocol today and the problem of untraceable attacks will be just as prevalent tomorrow. The largest attack observed (>1tbps didn't use amplifying UDP services at all).
BCP 38 would end this problem, full stop. ISPs complicit in what amounts to fraud deserve no sympathy. On most routers, ingress filtering is a one line option. Yet people like you promote breaking changes to UDP protocols as a half-baked solution that doesn't do anything to actually stop source spoofed DDoS attacks. It baffles me.
Victim blaming is wrong because you divorce someone's intentional decision to cause harm from their responsibility for the resulting harm. To take the classical example: It's not inherently wrong to tell people that they should avoid situations that empirically have an increased risk of being raped. It's only wrong if you then claim that the perpetrator is not responsible for what they did because the victim didn't heed that advice.
Now, whether ISPs are victims isn't really that clear. First of all, the DDoS packets consume bandwidth, which can cause increased expenses without corresponding increased income. Secondly, at least the very moment you expect them to implement BCP 38, they absolutely are. Implementing BCP 38 is additional work that's only required in order to handle the bad behavior of other people that the ISP is not responsible for, aka harm due to someone else's malicious actions.
But also, the comment that you responded to above was about DNS cache poisoning, not about DDoS. By randomizing DNS transaction IDs, you absolutely can make DNS cache poisoning harder, and I think the responsible thing to do is to protect against that attack vector. ISPs should be doing more to prevent spoofing, but that doesn't mean that it's responsible to leave your users open to spoofing-based attacks that you should be able to relatively easily prevent.
Implementing BCP 38 is additional work that's only required [..]
"Yeah, this guy stole your car by asking nicely for an extra set of keys from the manufacturer. Even though the manufacturer shouldn't have allowed him to do get a key, we're holding you responsible since you technically could have kept your car in a locked garage."
Also, the gp was taking about cache poisoning, but it was only exploitable in the presence of spoofed IPs.
Performing edge ingress traffic filtering of known-trunked node addresses (i.e. BCP 38) has a high likelihood of reducing the effectiveness of DDoS attacks.
People are operating completely sane services and anti-UDP warriors suggesting they change them because they appeared in a ddos is idiotic and harmful to the Internet. You can completely eliminate all UDP services and the untraceable DDoS problem won't go away, you've just eliminated some amplification avenues. The 1.5tbps attack didn't even use UDP so this "solution" is dead in the water and whoever is pushing it is misinformed.
Do you realize that every public DNS server in the world (root servers, recursive revolvers, Google.com NS servers, etc) can be used in an amplification attack? According to you we should name and shame every DNS developer, domain name owner, and operator.
We essentially couldn't have functioning DNS if we operated under your plan to 'fix this from both ends'. I suggest you read up on amplification attacks because I think you misunderstand how they actually work and think there is a way to 'configure' your way out of them.
This problem absolutely should not be "fixed from both ends". The right hand side (eliminate all amplifying UDP protocols) is massively disruptive (bye bye DNS) and doesn't actually fix the root issue (attackers just spoof TCP instead with a reduced amplification factor). The left hand side (BCP 38) requires fewer participants (ISPs vs server operators) and it completely solves the problem even without the slightest adoption of the stupidity on the right hand solution.
If you have ever suggested that a dns operator disable recursive resolving, you are perpetuating the problem by making people participate in an idiotic ritual that does almost nothing to improve the security of the Internet. It's the equivalent of restaurants not serving glasses of water to reduce the water shortage while farmers grow rice patties in a desert with essentially unlimited water rights.
See why trying to 'fix' this at the DNS level is stupid?
What? What crusade? I love UDP, use it every chance I get.
Who is "crusading" to end UDP on the Internet?
QUIC uses tokens for example.
That's why this approach of attacking UDP instead of spoofing is misguided.
Most of the fine people running chargen servers seem to be running the server from Microsoft services for UNIX package which is configured to send giant fragmented responses, and then they have a firewall that drops the first fragment. Thanks a lot guys.
Does not exist and if it did, it's the unspoofed traffic you should be worried about.
1) There is no evidence that the recent giant DDOS attacks on Brian Krebs used IP Spoofing. In fact, there is every reason to believe that they did not since the generators of the packets were low powered IoT devices. There is increasingly little reason for attackers to even bother with IP spoofing given how easy it is becoming to capture giant herds of low power IoT devices. The attackers don't care if some of their herd gets taken offline due to effective attribution.
Amplification/reflection attacks which will still require IP spoofing. What I'm curious about, and only time will tell, is how much IP spoofing will continue to play a part in lsrge DDOS attacks? Why bother spoofing IPs if your botnet herd is already large enough to bring someone offline?
2) Go and download CAIDA's Spoofer application. Test it and give them bug reports. I gave them one a few weeks ago.
This does not invalidate this presentation though. The recent IoT attacks are a new wave, but the point is:
- if it's not IP spoofing - you can track the botnet and eventually destroy it. I'm not saying it's easy or hard, but that it's technically possible.
- if the attack is IP spoofing, it's very hard to track it down and destroy the source.
In the days of Shodan, NO. Absolute no. 100/10 or 50/5 MBit/s household networking is becoming the norm in Germany, and other countries are way ahead of us Germans. Add in the fact that people with lots of (crappy) IoT devices are people who also have the money for high-speed internet connection...
Then throw in a couple of stro's at DCs with good interconnections, and you got yourself a niiiiiiiice huge botnet. You can do LOTS of damage if you get maybe ten or twenty servers with 10 GBit/s links under your control! (10 servers = 100 GBit/s, 10% of the Krebs attack IIRC)
Fiber is coming to the household. It's gonna bring the potential for DDoS to another order of magnitude. :D
Well. The IoT device will be limited by the bad Wifi or 100 Mbps Ethernet. Hopefully.
Of course toward the end of it I learned that I could have done this all with iptables, but I like my way better because I got to learn a lot.
This is just a smell emanating from another poor design, not a justification for IP spoofing. If you want to spoof addresses, do it in the privacy of your own VPN tunnel between the servers. Don't expose that sickness to the Internet.
First, I never said we identified the sensors by IP, but it was important that we record the sensor's IP for diagnostic information.
Second, IP spoofing was not done over the Internet, but rather only between our own servers inside our data center. Obviously doing this over the internet would not only be bad, but also highly unreliable.
In conclusion... well I'm trying to be polite.
If I remember right, we would only notice it when we were confused by packet captures, when we had strange performance problems, or when we'd discover that we suddenly had two single points of failure. :)
I hear you saying "just blackhole them until they figure it out," but it's not that easy. In many cases, the small regional ISP is the customer of a larger ISP, who is the customer of an even larger global ISP that you are connected to. You just see XXXgbps of traffic coming from your ISP, and have no idea which one (or more) of their customer's customer's are sending the traffic.
This would be like saying "just don't let the one guy that's going to pee in the pool swim." How do you know which of the 200 people in the pool actually peed?
This is a serious problem and it should be treated with serious consequences.
Sounds like a logic puzzle. I'm thinking a binary search would be the most efficient way.
I've also wondered why ISPs don't do more to shut down customers that are participating in a DDOS (at least for DDOS attacks where the source IP isn't spoofed)? I would be very happy if my ISP were to let me know that something on my network is involved in an attack.
I can tell you they're a real pain to convince to unblock you when you are 17 and have been a bad netizen. Which is a good thing.
Unfortunately this requires ISPs to actually be responsible for the data coming out of their network, including DDoS traffic. Once you stop the spoofing, ISPs and transit providers will soon figure out which ASs are problems and start blocking them etc.
With what hardware?
There is no excuse except laziness for not doing do.
https://tools.ietf.org/html/bcp38 (Note: This is from May 2000)
So if you're going to try to deploy something like Tor, the table stakes are that you're secure against wide-scale Netflow instrumentation. If Netflow is an existential threat to your privacy tool, you don't have a privacy tool that is ready for deployment.
This was debunked.
I don't believe tor can support the type of ddos the OP is talking about, of course.
If you could provide the source of this debunk it'd be appreciated.
No one claimed Tor was sending DDoS attacks -- plenty of other malicious traffic comes out of Tor however.
However, when I see tricks like in this talk using iptables with BPF bytecode to block SYN packet floods, I get completely humbled. I know that I know nothing.
... which you can conveniently buy on one of the CloudFlare-protected DDoS service websites! I know this point has been hammered to death before, but I find it curious that despite their strong advocacy of being content neutral and not removing a site unless they receive a court order, their Terms of Service explicitly allows them to shut off service based solely on their opinion:
"SECTION 11: INVESTIGATION
CloudFlare reserves the right to investigate you, your
business, and/or your owners, officers, directors,
managers, and other principals, your sites, and the
materials comprising the sites at any time. These
investigations will be conducted solely for CloudFlare’s
benefit, and not for your benefit or that of any third
party. If the investigation reveals any information,
act, or omission, which in CloudFlare’s sole opinion,
constitutes a violation of any local, state, federal, or
foreign law or regulation, this Agreement, or is
otherwise deemed harm the Service, CloudFlare may
immediately shut down your access to the Service."
They're not offering a safe harbor, they're offering a non-discriminatory service. Those are two different things.
"Solving" the "problem" of ip spoofing is only a benefit for centralized authorities and services. The loss of privacy is also serious. People advancing this idea are advancing it to better their commercial interests rather than the interests of individuals using the 'net.
Reverse Path Forwarding is best practice, and has been since more than a decade. The normal reason packets appearing on the wrong interface is a routing loop somewhere and this alone makes it worth it.
If that's indeed the endgame, it seems like a lot of work on the attacker's part to disable a company's servers for a bit, but maybe I"m missing something? The article did mention servers boiling, but that was likely hyperbole unless there's a way to physically damage servers with DDoS.
DDoS = No Website. No Website = no users viewing ads. No users viewing ads = No income.
Yes, that typically resolves quickly. Depending on your hosting agreement, you may have to pay for all the excess bandwidth that the DDoS bots used.
Now imagine if you are a bank, users want to be able to access their accounts/funds/etc. How long would you remain with your bank if you frequently couldn't access your information because they were being DDoS'd.
While it is very unlikely to cause hardware damage it can be extremely damaging to a company or service.
Service outages undermine customer trust and could drive users to your competitors. A company might also have a service level agreement with large financial costs associated with a service being down.
If you don't mind the service being down, then why have it? It going down will probably have some very negative effect.
First, one can spoof udp packets in the same way as tcp segments. Think syn or ack floods.
Second the Krebs ddos is said to generate over 600 gigs of traffic but it was not necessarily with fully established tcp connections. It's unlikely it was with valid tcp connections.
Let me give you a thought experiment: if you are flooded with 600 gigs of forged tcp packets, how do you know if the source ip's are spoofed or not? How do you know the ip in the src ip field belongs to the one that originated the packet?
Except for the latest and largest attacks: OVH @ near 1Tb/s in combined tcp_ack traffic on 2016/09/20 
When attackers have 150k+ comprimised hosts, with 1-30Mb/s each, there is no need for reflection/stealth/source spoofing/etc. At those levels, the attackers can just point them to an IP with 'normal' traffic and it's enough to be devastating.
 - https://twitter.com/olesovhcom/status/778830571677978624
When I think DDos, I envision thousands of compromised hosts all over the internet making requests to a target to consume resources.
To be honest, attackers are not very smart. They almost always use the same old attacks, attacks that are easily stoppable, and attacks against a single IP.
100gig coherent is easily turning volumetric attacks into old hat. Ending spoofed attacks today is a tactic that's 10 years too late.
This is why solving spoofing is such an important problem. Once source IPs become sticky, we can actually block based on them to truly stop the attack without taking down the service.
If the traffic is indistinguishable from normal traffic via L3/L4, then it's a layer 7 attack, which cannot be spoofed. You're not dealing with spoofing in a L7 attack.
The first D in DDoS stands for distributed. Even terabit sized attacks are easy to stop if they're sufficiently distributed (the attack is geographically dispersed enough to be diffused). It's when you get a terabit from a handful of AS's (like in S. Korean attacks) that you can feel pain. Even then, it usually only knocks out access in that part of the world, but not everyone.
IP's will not be sticky. IPv6 is showing why it's important to not be sticky. Ending spoofing isn't going to stop DDoS attacks. I've got 10+ years of experience on the subject, and I'll gladly answer any questions you might have.
A TCP SYN, or any UDP initial request indistinguishable from legit traffic of the same protocol at L3/L4, and both can be spoofed so I don't know where you got the idea that they cant be spoofed and are L7 only.
If you're "stopping the attack" through dispersion, you're not actually stopping it, you're just paying the price of absorbing it. I know this is common practice, but it's pathetic than an arms race is the state of the art in defense. It concentrates power into orgs like cloudflare, akamai, and Google (project shield) because the only way to participate is to use them.
I'm really impressed with this presentation, but I wish that one flaw could be fixed.
It's a technical talk. It's much better to hear from people who are being attacked regularly and have a track record of dealing with it rather than those who have no clue.
You can pay them, you can do it yourself, you can pay someone else, but at least you have options.
Except, why put that ridiculous meme in the middle of it? It's cringeworthy seeing an excellent technical presentation littered with such childish imagery.
(Not that I agree with this bastardization of "meme" to mean "silly image with text overlaid in capital letters", but unfortunately that is what everyone is calling these things.)
The term for that is "image macro" . An image macro represents a meme when the image is associated with a particular joke format, catch phrase, archetype, etc..