The problem is that the linux kernel can't process many simultaneous small connections.
Just to be clear:
- linux can easily transfer at 40 Gbps
- linux chokes at around 1M packet per second per cpu socket.
So linux can easily transfer at 40 Gbps one or few simultaneous flows
But! It can't transfer several flows at 40 Gbps.
The bottlebeck is the number of packets per seconds it can inspect.
So if you are sending 100 big transfers, you will reach 40 Gbps.
If you send 5M small transfers, linux will die.
This is why netmap is handy.
It offloads the packets from linux directly to the app
Bbc is sending xM small requests per second, hence millions packets per seconds.
In conclusion, netmap is good if you need a lot of small simultaneous connections.
I reached 40M connections on chelsio 40 Gbps nic using netmap on FreeBSD 11.
which is why kernel bypass mechanisms f.e. dpdk/netmap/... etc. are getting some/lots of traction. fwiw, using dpdk (on x86-64 hw), i have had no problems with pushing minimal 64b sized packets for large (>250k) number of flows at line rate...
The key point here is that a netmap driver bypass the kernel and therefore open the door to many millions packets per seconds or millions requests per seconds. Not many Gbps