My ideal tool would let me switch a knob to turn on io profiling and then would let me drill down into the trace, show what processes caused the io, was the io like (contiguous vs seek heavy, fsyncs, etc). A second need is report what kind of io a process incurred during its runtime, a bit like "time".
I guess much like "perf" now but for io.
For all I know all the kernel pieces are there already. We do have some newer tools like iotop/iostat that are small steps forward. But it's pretty quiet.
pidstat (part of Sysstat, "apt-get install sysstat" on Debian/Ubuntu) does this pretty well.
# sudo pidstat -d 1
11:21:54 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command
11:21:55 PM 855 0.00 7.92 0.00 kjournald
11:21:55 PM 8134 0.00 79.21 0.00 rsyslogd
11:21:55 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command
11:21:56 PM 829 0.00 28.00 0.00 kjournald
11:21:56 PM 8134 0.00 60.00 0.00 rsyslogd
11:21:56 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command
11:21:57 PM 8134 0.00 104.00 0.00 rsyslogd
11:21:57 PM PID kB_rd/s kB_wr/s kB_ccwr/s Command
11:21:58 PM 8134 0.00 56.00 0.00 rsyslogd
Still, it's pretty far from DTrace's iosnoop script (on OS X, do "sudo iosnoop" to see it). Something I dearly miss on Linux, but I don't know what the situation is with SystemTap or DTrace these days.
You also have access to the dtrace probes.
vmstat is essential in it's own way, the run queue is critical in threaded environments like weblogic with osb/soa where a default install on a 2 vCPU instance won't perform.
Also; dtrace for linux and osx.
It uses colors, so it's a bit easier to interpret at first sight.
It's a simple "emerge dstat" away in Gentoo.