You're stating it as if it were a shortcoming of PulseAudio. PA is inadequate for most audio work, because the latency is too high -- which is a feature.
Latency vs. CPU usage is always a compromise. To achieve low latency, you need very small buffers for "rendered" audio, and lots of well-timed copy operations to the audio hardware. To just play some audio files efficiently, you want large buffers, because then fewer copy operations are necessary.