Yeah, I had the same thought when the discussion came around last time. I actually took the source to the simulations (thanks for sharing, rapgenius!) and did some experiments on how random routing performs if it's routing to backends that can handle various #s of concurrent requests.
> and let the operating system do all the complex scheduling.
So Heroku could spawn workers with a $FD environment variable instead of $PORT and the "complex scheduling" done by the OS _is_ the second routing layer.
But really, they could still do a second level routing even outside a single OS as the scale of the distribution is much smaller, so having the routing mesh be aware of the worker availability seems feasible again.