
Front-End Servers and Client-Side Random Load Balancing - dsr12
http://highscalability.com/blog/2016/1/4/server-side-architecture-front-end-servers-and-client-side-r.html
======
scurvy
I don't understand why the author says server-side load balancing is expensive
or must be done with expensive hardware. Yes, you can use hardware accelerated
boxes to speed up things like SSL termination and HTTP compression, but
they're not required. Two alternatives:

1) IPVS with DSR. You'll get about 1M PPS through each "load balancer".
Regular, cheap Linux boxes do well here. You can get fancier here, but a pair
of these would scale most games just getting started.

2) Run a dispatch server. This is what we used in MSN Messenger. I'm not going
to go too deep into the details, but the gist of it is that we had two groups
of user facing servers: connection servers (CS) and dispatch servers (DP).
Connection servers is where your client TCP connection terminated and was
responsible for updates to your client. The DP server merely "dispatched" you
to a CS upon initial client connect. The DP had no role other than seeing
which CS servers were alive and sending you to one of the least loaded. The CS
servers advertised their existence and load on a multicast address. Yeah, we
had a hardware load balancer in front of the DP servers, but we very easily
could have used a software implementation. We were part of Hotmail so there
was loads of networking hardware to use.

Client -> Cisco CSM -> DP server -> redirect to CS -> CS

After that, the CSM and DP server were out of the loop and everything went
to/from the CS directly. Might not be super elegant, but it worked really
well.

------
padobson
If you can assume that any given client will consume a relatively set range of
server resources to the point where random load balancing evenly distributes
load for large numbers of clients, then why not just have the front-end
servers return some sort of "server's full" header and tell the client to try
another? Are the extra requests so expensive? Or is using the random
distribution simply a case of "This works, so why bother with non-random
methods?"

~~~
LASR
Multiple network round-trips to find a 'good' back-end is undesirable. If your
client is browser-side javascript, for example, then these can directly affect
responsiveness and time to page usable.

Random is significantly easier and cheaper to implement. As you scale your
service to multiple back-ends for the first time, you pragmatically start off
with a random client-side load balancer - because it is cheap and simple.

Load balancing is one of those things in your product that once working to
some minimum extent, nobody cares about improving. Swapping out the client
side load balancer for a 'real' load balancer is the last thing on team's
mind. So, you'll see a very large number of mature products continuing to use
them.

Last point, random-client side load balancing provides a true solution for
SPOF where as in the other methods, you still have at least on SPOF somewhere
in the system.

------
guidedlight
The HTTP/2 specification which is now supported by every major browser, has
built in support for Client-Side Load Balancing via the ALTSVC handle. It can
also be used for transparent failover, and can do all this without touching
DNS.

There are heaps of other advantages by moving to HTTP/2, including an average
performance boost of 15%. It also transparently downgrades to HTTP/1 for
unsupported clients.

------
erichocean
If you're implementing randomized load balancing, consider using two-choice,
randomized load balancing by default.[1]

[1]
[https://www.eecs.harvard.edu/~michaelm/postscripts/tpds2001....](https://www.eecs.harvard.edu/~michaelm/postscripts/tpds2001.pdf)

