Hacker News new | comments | show | ask | jobs | submit login

Many people don't understand how or why the linux kernel is the bottlebeck at speed higher than 10 Gbps.

The problem is that the linux kernel can't process many simultaneous small connections.

Just to be clear: - linux can easily transfer at 40 Gbps - linux chokes at around 1M packet per second per cpu socket.

Thats right.

So linux can easily transfer at 40 Gbps one or few simultaneous flows

But! It can't transfer several flows at 40 Gbps.

The bottlebeck is the number of packets per seconds it can inspect.

So if you are sending 100 big transfers, you will reach 40 Gbps.

If you send 5M small transfers, linux will die.

This is why netmap is handy.

It offloads the packets from linux directly to the app

Bbc is sending xM small requests per second, hence millions packets per seconds.

In conclusion, netmap is good if you need a lot of small simultaneous connections.

I reached 40M connections on chelsio 40 Gbps nic using netmap on FreeBSD 11.

> linux chokes at around 1M packet per second per cpu socket.

which is why kernel bypass mechanisms f.e. dpdk/netmap/... etc. are getting some/lots of traction. fwiw, using dpdk (on x86-64 hw), i have had no problems with pushing minimal 64b sized packets for large (>250k) number of flows at line rate...

This isn't specific to Linux. PPS is a limiting factor for network hardware vendor's gear as well. Lots of ~100 byte packets will bring pain to Juniper and Cisco products as well.

Thanks, I've always wondered under which conditions BSD networking performs better compared to Linux. It's the number of simultaneous streams as opposed to raw bandwidth for fewer streams.

Well BSD kernel will also choke at more or less the same limits as linux kernel.

The key point here is that a netmap driver bypass the kernel and therefore open the door to many millions packets per seconds or millions requests per seconds. Not many Gbps

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact