I seriously considered moving to SmartOS as ZFS and Zones and likely Dtrace as these features that would make my job, that largely comes down to organizing running software on machines and debugging problems, far easier..and would allow me to use the machines to better degree but in reality it's not going to happen.
Nobody is familiar with Solaris userland. I'm not a sophisticated and educated systems engineer at Joyent I'm just a stressed guy trying to fix problems. Unfortunatly Linux is pretty good at making it work because a lot people are in a similar situation and someone will fix it for me.
I just don't have the time and knowledge and energy to e.g. fix native Hadoop libraries in the ecosystem to build with another libc or make my own or other applications able to run without some Linux specific crap..
That beeing said I really thought about pushing SmartOS/Solaris but as a lone fighter It would be suicide in a world where everyone knows apt-get install <whatever> and get his shit done in a reasonable way..
Maybe it's something for specialised application and not academia
I've came pretty far with just strace and perf top and most problems I had in my own application where better analyzed by valgrind and kcachegrind or massif and the visualizer...
I've also thought about FreeBSD and while pkgng is really great it's a similar problem.. I'm stumbling upon bugs or untested things and I'm unable to contribute time to fixing them.
I don't run anything big on FreeBSD and ZFS. I have not experienced problems on a raidz2 ZFS fileserver that runs FreeBSD except that disks drop out quite randomly but I've yet been unable to pinpoint that and it's likely that these are hardware issues as the system runs on budget hardware.
Sorry if my comment spreaded FUD.