I make heavy use of HAProxy and use the silent-drop capabilities on my own personal sites. In addition to stick tables, I also use it heavily with acl's that pattern match on common attacks and enumeration.
My favorite stick table uses http_err_cnt to block enumeration attempts.
stick-table type integer size 127k expire 30m store http_err_cnt
acl abuses sc0_http_err_cnt gt 6
http-response silent-drop if abuses
acl BOTS src -f /opt/.ha/bot.txt
acl BOTS hdr_sub(user-agent) bash url get lack ing team oog lurp ook rawl bot map dback ltx pider scan bmast robe ithub hatweb ansci rchiv null isco =
acl FILES path_end .html .pl .jpg .jpeg .gif .png .gz .xz .7z .zip .rar .m3u .ogg .mp3 .mp4 .avi
acl FILES url_sub /post /upload /images
acl PROT hdr_sub(host) site1 site2 site3
http-request silent-drop if BOTS FILES PROT
There's still the fundamental limitations of incoming bandwidth, system pps limits, etc., etc.
Having the ability to protect against more focused attacks with HAProxy could be very useful in those few instances where volumetric attacks are not being used.
At the moment, I'm using AWS and Elastic Load Balancer so, I'm just thinking HAProxy just doesn't fit well within the existing network layout.
Additionally, i'm not sure if I should just be solving this with a simple nginx/redis lua script. If you have any recommendations I'd love to hear how other people solve this rate limiting/throttling problem.
Even if HAproxy isn't suitable, I'd like to hear the "HAProxy" solution for it...
You can put HAProxy directly on a box running another service to provide protection/reporting if you don't want to replace or add HAProxy to your ELB's (though sometimes that just adds too much complexity), we also talk about using HAProxy and ELB in our AWS blog post (https://www.haproxy.com/blog/haproxy-amazon-aws-best-practic...).
It has a section on rate limiting. Specifically check out the section which starts with the sentence: "Optionally, instead of a daily limit as used above, you can also do it based on the rate of the requests."
And coming back to the topic of slow requests, what if an attacker tries a slow POST request, sending the first part of the request at a normal speed until it reach the size of the http-request-buffer, and then sending the rest very slowly? haproxy would let the request pass and the backend will be kept busy.
I'd love someone to write about that. Everything is always about HTTP/S
There's a few classes of effective DDoS:
A) volumetric traffic/packet counts: the only effective thing here is to have tons of bandwith or use a service that does. Some of the UDP reflectors out there have a very high amplification rate. Null routing can absorb the bandwidth, but smarter attackers will notice when you move your service and change the target.
B) application related, but not application specific. Things like slowlaris to hold connections, or just https floods to use all your handshakes/second. Some of these you can filter, but you sometimes just need more machines to process everything. Apparently HAProxy can help with slowlaris by detecting and dropping slow connections.
C) expensive requests / processing at the application level. If you have a public endpoint that takes 10 seconds to process a request that takes effectively no time to generate, that's definitely a potential thing to be attacked, and that's something HAProxy can definitely help with.
D) Issues in the IP/tcp stack. Sometimes there's gray areas or infrequently used corners of the processing that are very expensive if used frequently (ex: IP fragment reassembly). HAProxy won't help there.
If HAProxy (or whatever) can help you with low bandwidth DDOS, I think that's still pretty useful.
The problem is if your attacker is sending you more traffic than your incoming bandwidth. Packets will be dropped, and in most cases you won't be able to control which ones. Depending on how the other side is configured, what packets you do get could be highly delayed. That means actual connections to you are likely going to see a lot of retransmits to you as well as from you. It's possible to still make some progress in these conditions, but not very much, processing power won't help.
If DDoS protection isn't the point, what is?
I don't think it's discussed enough in our circles the various aspects of the internet that are more or less broken. Like how easy it is for anyone to take you offline. How easy it is to spoof IP addresses. How useless IP address blocking is. How we demand infinite bandwidth for a low, fixed, monthly price yet don't want to be on the hook when our toaster is DoSing our neighbor and causing real financial damage.
But at the same time we share these little haproxy/fail2ban tips that don't work under actual threat, and then we lament that people use services like CloudFlare instead of talking seriously about how we depend on the free services of large companies, whether it's CloudFlare's DDoS protection or Google's reCaptcha, to prevent real abuse.
The heaviest and hardest to maintain features in these environments are the fat stuff that customers want (WAF, monitoring, UI, config versioning, etc). But basic protection is trivial if you can afford the bandwidth.
Assume that you have 2 haproxy instances and a client makes 2 requests and each request lands on a separate haproxy instance. With peers, the request count would sync as "1" but with the stick table aggregator the request count would sync as "2".
The principle is that there will be one table per metric and per node, and each node will track values using its own table. This way each table is written to by only one node, and all nodes get all the table updates. They are then free to watch other nodes' values in their respective tables.
Using the arithmetic converters it's even possible to add or average values between all tables, but then this quickly becomes complicated, as for example you can't easily take into account the number of online nodes in your operations.
This is why this is mostly workable with two nodes and not really much more. In this case of two nodes, you either use your own value or the average operation if you can ping the neighbor.