what kind of problem were you trying to solve that you found sizing a thread pool to be difficult? generally when I've worked on high performance server code I've been coding with a target machine in mind, so it's more a matter of mapping the thread pool size to the resources available on that machine. but I'm interested to hear about circumstances where it wouldn't be easy.
If you have downstream nodes which may have large amounts of latency in some scenarios, then you may need a huge thread pool.
If you add a huge thread pool, and then those downstreams don't have a large latency, then you end up accepting a huge amount of work and then are CPU starved.
So in order to correctly size your thread pool, you need to understand all your downstream latency, and adapt to it.
Compared to an async runtime, which just handles this scenario, it's very painful.
Even if you get this roughly right, the scheduler is very unhappy when you have lots of threads - it tends to make incorrect scheduling decisions.
You have a threadpool with X threads. You dispatch Y tasks. X of them run for 5 minutes. That means the remaining Y-X tasks are delayed by 5 minutes despite low CPU utilization.
