Even a simple button involves half a dozen threads in AmigaOS between the drivers, the handlers "baking" raw events from the drivers into something higher level, the Intuition thread processing the button presses into events related to a specific button etc.. It affects total throughout but increases responsiveness.
I think that if the OS provides responsiveness, and some key apps do, people will demand it. That is what happened with AmigaOS. You didn't get away with making a system sluggish because you'd get compared to apps that were extremely responsive.