
Snabb: 100 Gbit/s pure software switching using Lua (2019) - pdmccormick
https://github.com/snabbco/snabb
======
myrandomcomment
I would love to see an IXIA or Spirent line rate RFC2544 test with an imix
packet sizes. Most servers and OS stacks cannot do line rate 100G/bits today
without a ton of tweaking. Netflix has a few blogs on it. In my current
company we moved our NAS to 2x100G but the build servers at 10G and 25g
(Ubuntu) cannot max a link in their normal operation (which includes pulling
4g VMs). There is some low hanging fruit here tweeting wise we can do but..

The NAS is TrueNAS with SSD and all the ZFS goodness, ZIL, etc. I can built
hand crafted iperf streams between it and another 100g server that hit line
rate (including overhead).

The reason for the 100g is there are racks and racks of BS that pull down
their images so aggregate we can hit 100g with a fully non-blocking Clos
fabric.

~~~
yjftsjthsd-h
> The reason for the 100g is there are racks and racks of BS that pull down
> their images so aggregate we can hit 100g with a fully non-blocking Clos
> fabric.

It may not pan out in practice, but in theory isn't that the perfect usecase
for p2p image distribution?

~~~
dicknuckle
could probably do it with BTsync if you don't want to build your own. Or
there's also Syncthing which is more open source

~~~
yjftsjthsd-h
I was thinking ipfs, but yeah there's lots of ways to skin this cat.

------
tpmx
Something "fast" named "snabb", from Switzerland. I bet they get their kicks
out of triggering Swedish people.

Snabb is Swedish for fast. Switzerland is often confused with Sweden and vice-
versa, stereotypically by the under-educated kind of americans.

Grr. :]

~~~
kristoffer
Well the creator of Snabb lives in Sweden.

~~~
patrec
He used to, but I believe he now resides in Switzerland.

~~~
lukego
Moved back to Sweden recently, as it happens :)

~~~
tpmx
Sorry about the timing :)

------
7kay
Interesting, thanks for sharing. It's always nice to see Lua in the wild. This
headline immediately reminded me of this[0] talk from CCC 2018 comparing
different high level languages for writing userspace network drivers.

[0]:
[https://media.ccc.de/v/35c3-9670-safe_and_secure_drivers_in_...](https://media.ccc.de/v/35c3-9670-safe_and_secure_drivers_in_high-
level_languages)

------
dang
See also

2017:
[https://news.ycombinator.com/item?id=13775773](https://news.ycombinator.com/item?id=13775773)

2014:
[https://news.ycombinator.com/item?id=8008963](https://news.ycombinator.com/item?id=8008963)

------
tr33house
Just out of curiosity, what kind of speeds can be achieved by a pure software
switch using say C or pure assembly?

~~~
p_l
C probably won't give you much, difference. Assembly, carefully tweaked, might
- the kind of stuff like Quake issuing FDIV every 16 pixels in software
renderer, where the assembly intimately matches various aspects of
microarchitecture.

~~~
BubRoss
What are you basing this on exactly?

~~~
p_l
Despite years of propaganda, C is not well matched to CPUs currently in use
(quite opposite in places), and the typical optimizations don't necessarily
work when dealing with external I/O that you need to do in a switch.

Essentially, even if you write in C, reaching higher speeds will involve using
"assembly masquerading as C" rather than depending on compiler optimizations.

Also, Snabb uses LuaJIT, which already generated quite tight code, so the
performance gap that I suspect some imagine just isn't that wide.

~~~
fit2rule
>C is not well matched to CPUs \+ >depending on compiler optimizations \+
>Snabb uses LuaJIT .. quite tight code

==

You can write great C-based systems and avoid assembly if you a) know, always,
what your compiler is doing and b) know, always, what your CPU is doing...

~~~
exikyut
I'd be interested to learn about significantly complex, nontrivial systems of
say 100K to 1M LOC scale that require reasoning about every single instruction
from the perspective of every other instruction, in order for the system to
work.

~~~
benou
You do not do that. Instead you optimize the short, critical part.

Of course it does not apply to everything, you need a few hotspots, but it is
quite common: audio/video codecs, scientific computation, games, crypto... And
even networking.

~~~
fit2rule
And with Lua (and C, and C++) its pretty easy to manage the complexity. Just
put things where they belong.

------
sandGorgon
could someone explain where in the stack does Snabb sit vs something like BPF
? [http://www.brendangregg.com/blog/2019-12-02/bpf-a-new-
type-o...](http://www.brendangregg.com/blog/2019-12-02/bpf-a-new-type-of-
software.html)

it seems that Snabb uses pflua at its core - which claims to be faster than
BPF (
[https://github.com/snabbco/snabb/blob/b9da7caa1928256a3b4908...](https://github.com/snabbco/snabb/blob/b9da7caa1928256a3b490867e9bc62dfbcd21ede/lib/pflua/README.md)
)

~~~
corysama
BPF's claim to fame is that it is confidently secured enough that it can be
used to run user-provided code in kernel space.

Snabb side-steps the kernel by providing the performance aspects of kernel-
like access while staying securely in user space.

~~~
pas
Currently that user is almost always the administrator (root). BFP is more
safe than secure, if that distinction makes some sense.

So it's great because the eBPF verification makes sure that whatever the user
loads doesn't blow up the kernel immediately, it's very restricted (no
infinite loops, so not Turing complete, can only call predefined API
functions, etc). But it still gets JITed into machine code, and still has
access to a vast trove of kernel gadgets and data. So it's like a power tool
that was designed to be almost impossible to cut your own limbs off with.

But it's still a power tool, it still not something you give to any random
passer by.

~~~
wahern
eBPF has had many exploits, the most recent of which was just a few months
ago: [https://www.thezdi.com/blog/2020/4/8/cve-2020-8835-linux-
ker...](https://www.thezdi.com/blog/2020/4/8/cve-2020-8835-linux-kernel-
privilege-escalation-via-improper-ebpf-program-verification)

Original BPF was safe by design (no verifier needed) but slow and far more
restrictive, and even then there were exploits over the years. When eBPF and
then eBPF JIT became a thing it was entirely predictable what would happen.

