> The downside of the kernel is that there's no way around it. You can't "opt out" of any of its decisions.
Well, you can sidestep most of it. For example, re the c10k/c10m problem, you can the transport protocol impl
to userspace. See example here how they did it with SCTP: http://www.cs.ubc.ca/labs/dsg/mpi-sctpuserspace_ICCCN2012.pd... - you could do the same with TCP if you don't care about having a TCP-enabled kernel.
The other example of course is all the protocols built on top of UDP.
This still has the NIC driver shoveling at least the raw IP packets via the kernel driver, but you could sidestep that too by using the hypervisor interface to route the NIC pci space and interrupts to your program (like Xen uses).