Hacker News new | past | comments | ask | show | jobs | submit login
Front-End Servers and Client-Side Random Load Balancing (highscalability.com)
45 points by dsr12 on Jan 5, 2016 | hide | past | web | favorite | 6 comments

I don't understand why the author says server-side load balancing is expensive or must be done with expensive hardware. Yes, you can use hardware accelerated boxes to speed up things like SSL termination and HTTP compression, but they're not required. Two alternatives:

1) IPVS with DSR. You'll get about 1M PPS through each "load balancer". Regular, cheap Linux boxes do well here. You can get fancier here, but a pair of these would scale most games just getting started.

2) Run a dispatch server. This is what we used in MSN Messenger. I'm not going to go too deep into the details, but the gist of it is that we had two groups of user facing servers: connection servers (CS) and dispatch servers (DP). Connection servers is where your client TCP connection terminated and was responsible for updates to your client. The DP server merely "dispatched" you to a CS upon initial client connect. The DP had no role other than seeing which CS servers were alive and sending you to one of the least loaded. The CS servers advertised their existence and load on a multicast address. Yeah, we had a hardware load balancer in front of the DP servers, but we very easily could have used a software implementation. We were part of Hotmail so there was loads of networking hardware to use.

Client -> Cisco CSM -> DP server -> redirect to CS -> CS

After that, the CSM and DP server were out of the loop and everything went to/from the CS directly. Might not be super elegant, but it worked really well.

If you can assume that any given client will consume a relatively set range of server resources to the point where random load balancing evenly distributes load for large numbers of clients, then why not just have the front-end servers return some sort of "server's full" header and tell the client to try another? Are the extra requests so expensive? Or is using the random distribution simply a case of "This works, so why bother with non-random methods?"

Multiple network round-trips to find a 'good' back-end is undesirable. If your client is browser-side javascript, for example, then these can directly affect responsiveness and time to page usable.

Random is significantly easier and cheaper to implement. As you scale your service to multiple back-ends for the first time, you pragmatically start off with a random client-side load balancer - because it is cheap and simple.

Load balancing is one of those things in your product that once working to some minimum extent, nobody cares about improving. Swapping out the client side load balancer for a 'real' load balancer is the last thing on team's mind. So, you'll see a very large number of mature products continuing to use them.

Last point, random-client side load balancing provides a true solution for SPOF where as in the other methods, you still have at least on SPOF somewhere in the system.

Your idea actually only makes sense if load is not equally balanced. If it is equally balanced that means all servers will become "full" at approximately the same time. Unless you exchange information about load between servers (which is difficult) this means a client will have to try a large number of servers until it finds one that is not "full".

The HTTP/2 specification which is now supported by every major browser, has built in support for Client-Side Load Balancing via the ALTSVC handle. It can also be used for transparent failover, and can do all this without touching DNS.

There are heaps of other advantages by moving to HTTP/2, including an average performance boost of 15%. It also transparently downgrades to HTTP/1 for unsupported clients.

If you're implementing randomized load balancing, consider using two-choice, randomized load balancing by default.[1]

[1] https://www.eecs.harvard.edu/~michaelm/postscripts/tpds2001....

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact