Kudos to the kernel developers who have tried nearly everything under the sun to make performance of multi-core, multi-task, general-purpose machines satisfy everyone at the same time. Your efforts are not lost on those of us who understand just how difficult that is.
In the late '80s I worked at Interactive System Corporation, the company that AT&T contracted with to do the official System V Release 3 port from the 3B2 to Intel processors. I was initially working on the port to the 286.
The 3B2 was a paging system, which did not really fit well with the 286, so it has kind of hacky. We never were able to get the scheduling to behave reasonably. Running 10 CPU bound tasks, for example, would result in one getting about 85% of the CPU, another getting about 10%, and the other 8 splitting pretty evenly the rest. It would do this for a while and then would start massively thrashing swap for a couple minutes, and then would settle back down to the 85/10/* but with different tasks getting the 85 and 10.
We were still stuck on figuring that out when AT&T came to their senses and realized that no one actually wanted System V Release 3 on a 286 and we dropped it. At most what they wanted was to run their 286 Xenix and 286 System III binaries on their 386 System V Release 3 systems, and I got moved over to working on the emulators for those, which was much more pleasant.
(Besides multicore, another thing that I would guess makes it all much harder for today's kernel people is caching. Caching on the 286, and even 386, was a lot simpler than it is today. We were still in the days when systems were primarily getting faster because clock speeds were going up, not because they were being smarter. Now I think they get a lot of performance from very good use of resources, such as cache, and it is a lot easier in kernel software therefore to blow performance).
I'm actually kinda curious about the details on Porting System V, the 3B2 is a machine I find super interesting.
And there goes a large slice of all wasted engineering effort ...
Very cool! Are there any real-world applications that use non-default/custom schedulers?
Then you have some other RT Linux versions and VM schedulers in Xen.