Hacker News new | past | comments | ask | show | jobs | submit login

Isn't it possible to just set the haproxy maxconn to a slightly lower value than what the backend can deal with, and then let the reverse proxy retry once with another backend? Or even queue it for some hundreds of milliseconds before that?

This way you avoid overloading backends. Also, haproxy provides tons of real-time metrics, included the highwatermark for concurrent and queued connections.

What you suggest is what was followed by a large-scale distributed system I worked on. Your suggestion is also in-line with Google' SRE book section on Handling Overload: https://landing.google.com/sre/sre-book/chapters/handling-ov...

> Mandate that batch client jobs use a separate set of batch proxy backend tasks that do nothing but forward requests to the underlying backends and hand their responses back to the clients in a controlled way. Therefore, instead of "batch client → backend," you have "batch client → batch proxy → backend." In this case, when the very large job starts, only the batch proxy job suffers, shielding the actual backends (and higher-priority clients). Effectively, the batch proxy acts like a fuse. Another advantage of using the proxy is that it typically reduces the number of connections against the backend, which can improve the load balancing against the backend.

Other chapters on Load Balancing, Addressing Cascading Failures are related too.

Aaand this is what happens when one comments before reading all the article...

Applications are open for YC Winter 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact