Which is actually the case. In fact, audio is the best examples where everything can plug into almost everything. padsp (if I'm not mistaken - not at the prompt right now) does just that.
padsp is a hack that intercepts the open() calls to /dev/dsp - if things were done better opening /dev/dsp would just work without any wrapper, even with statically linked applications.