
Preventing server overload: limit requests being processed - jsnell
http://www.evanjones.ca/prevent-server-overload.html
======
eapotapov
just a few non-academic but tech remarks:

1\. when you limit the number of requests by just dropping them there is still
a huge chance to get a lot of troubles: each HTTP request from the browser now
can lead to dozens of additional requests, ajax/push etc. if some of them
would be dropped - the user might get a broken page which sometimes even worse
than just "try to reload" one. so there should be some smart balancing / smart
rules for the packets dropping

2\. from my experience there are two situations when the web app gets
overloaded: excessive cpu usage, excessive ram usage (when we talk about
databases it's disk as well but it's a different story)

excessive cpu usage have its own scheduler/queue which usually works pretty
well and doesn't have any side affects when you just rebalance/scale the
system. assuming that your web app responds in 200ms and the cpu is having
300% of usage it's just going to be 600ms for you in a worst case scenario.

excessive ram usage is much worse as you can't easily "oversell" it - either
you have swap and the system will start writing to the disk either some of
your processes will get killed by out-of-memory killer (and most likely it's
going to be something that's necessary to keep the app working). accessing the
disk instead of memory is very time consuming (10-100x if you have good ssd,
thousands of x if you have regular hdd). even if you stop the traffic - your
system processes might still use swap memory (it takes time for linux to move
them out of swap memory), so you probably won't be able to access the system
for a long time (20-30 minutes more for instance). this means you should be
very picky at what you set by the threshold for the limit - it couldn't be
just some average value, it should be the limit of the thing that brings you
most troubles when overloaded.

