I am not sure. But from the outside, it looks like what Prometheus does behind the scenes.
It seems to me that Prometheus works like that because it has a limit on latency time around 10s on some systems I worked.
So when we had requests above that limit it got all on 10s, even though it could be higher than that.
Interesting.
Not only that, but it's also fairly potent, allowing you to pass variables from one request to another, transform response, inject .env vars etc.
If all of your team is using Jetbrains products, you can also add the http files to the version control so everyone could have the requests ready to use.
I liked the approach. I did not know it.
But what are the advantages against taking the least occupied server/bin always?
I have seen good improvements with nginx load balancing when changing its configuration to least_conn, that chooses the server with least connections.
> what are the advantages against taking the least occupied server/bin always
The other answers are saying for algo reasons. Sure, but, pick two is even more better for uptime reasons.
In serving at scale, quite often the "least occupied" server of all of them is one with an error causing it to have least load/connections/whatever.
By starting with a random two, you pick among probably healthy servers, and then the least load.
> good improvements with nginx load balancing when changing its configuration to least_conn, that chooses the server with least connections.
nginx doesn't know why your app is failing to keep connections open long enough to serve them, least_conn between two random results in far fewer 100% "outages" when a single box or subset of boxes among 10s or 1000s is bad.
Not my field but I'm guessing when you have thousands of servers it can be quite the overhead to keeping track of the load of each one for each request.
The proposed algorithm only needs to consider two, and from what I can gather should work in parallel with multiple load balancers without coordination.
It’s more costly to determine the server with absolute least connections since you have to keep track of the ordering of the servers by how many connections they have.
Short version: it's robust to staleness and concurrency, which is important in distributed settings ("find the best" is hard to do when "best" is a moving target).
In large enough systems, it becomes prohibitively expensive to keep consensus of the state of all members.
Sure with 10 or so hosts it’s trivial to select the least busy one but at the scale of 1000s of hosts you begin to run into a ton overhead just to decide where to forward a request.
As many others have said, O(N) for each request vs O(1) while avoiding hot spots.
In my experience, I would say most people have problems with exactly this.
They don't even think about the possibility of noise/interpretation in the communication.
and they never stopped to think about it the way you did.
If you (+above commenter +any other HN per users) want a good Wikipedia article, the metalanguange/metacommunication ones do a great job of explaining how this works in CS friendly terms - the connection to meta programming is a great aside.
I saw some systems that used single line, even in these cases. It had an ephemeral/transitory account to help on this.
And to map it together, all these entries would be related to a single transaction.
It seemed a nice solution for me.
I think we only say things about the cost of time for leisure activities when it's not worth it. Example, I heard many times people saying about a bad movie: wow, I wasted 2 hours watching that
I've heard google would guess with bounce rate. Or another way, if the user clicks on LinkedIn website A, after a few moments keeps trying other linksw/related search. It would mean it was not valuable.
This works for me with workout. I commit to workout 10min everyday. Sometimes I and up doing more like 40m, sometimes only 10m, and it's ok.
And of course, I reduced the start energy for it. If it's only 10m, probably I will do it at home with some equipment.
