Not true. There are plenty of userspace TCP stacks in production today, just like there are also userspace IPsec stacks. UDP is also an option if TCP is too complex for your tastes.
If you believe that, you don't understand TCP, I recommend reading the actual RFC (RFC 793) and the IP RFC if necessary.
A port identifies a process, an IP address identifies a machine. The machine receives the packet, and then passes it to the process, since it's the machine that needs to receive the packet before routing it to the process, it's the kernel with ring 0 privileges that needs to process the packet and map the port to the process.
There's nothing stopping you from giving each stack its own IP, making the kernel only responsible for routing.
I don't think it's a good idea to put more of the stack than necessary into each individual program though, because then you're reliant on each program implementing everything correctly and you have to worry about bugs and security issues in every single program you're running rather than just in the kernel.
Considering that even something as simple as opening a socket (which involves calling one function to give you three numbers which you pass unchanged to a second function) is largely a clusterfuck in existing programs, I don't trust them to implement the whole TCP stack.
The performance hit for running an emulated CPU would be significant, to the point that it wouldn't really be the same argument.
I was thinking more along the lines of network namespaces -- although actually, I should have been thinking of TUN interfaces. Any packets sent into one of those are delivered to the attached program, which can do what it likes with them. Those are usually used by e.g. OpenVPN for L3 things, but nothing stops a program from handling the L4 headers.
You might want to read about exokernels. The kernel knows only enough about networking to pass the packet to the correct userspace process; it has no TCP machinery.
