
Google NetStack: IPv4 and IPv6 userland network stack in Go - yarapavan
https://github.com/google/netstack
======
beagle3
A search for "ledbat" in github doesn't find any reference, which is a shame.
Apple are the only one with ledbat support in the kernel[0] afaik, and it's
functionality that's sorely needed -- it's ridiculous that there's no way to
say "data on this connection is not urgent, only give it bandwidth as long as
it doesn't interfere with other connections". Some programs, e.g. rsync, let
you limit bandwidth - but that's wasteful; I want this 2TB rsync job to be
downloading at full speed at all time I'm not doing anything else.

[0] yes, this is userland, but it's for fuchsia, so "kernel functionality" as
far as its users are concerned.

~~~
vitus
From my brief look into ledbat, it seems like it serves a similar role to BBR,
which is in the Linux kernel now.

Either way, it looks like the congestion control is just New Reno.

[https://github.com/google/netstack/blob/edcfca33ca347f59a139...](https://github.com/google/netstack/blob/edcfca33ca347f59a1393986a5d216a5f183ae28/tcpip/transport/tcp/snd.go#L344)

Also, this isn't an official Google product, per the disclaimer; just some
code that happens to be owned by Google. Where are you seeing the reference to
Fuchsia?

edit: oh, I see. Fuchsia codebase has references to this netstack.

~~~
slackingoff2017
LEDBAT is similar but works on L7 instead of L4 and needs support on both
ends. BBR is excellent. It's the first congestion control strategy to really
focus on fairness rather than trying to send packets as fast as possible.

Combined with fq-CODEL, BBR finally offers a way to load balance traffic on
your NAT without allowing someone to monopolize bandwidth

~~~
kevin2r
Wow this is awesome, It bothers me that one computer on a LAN making a big
download, takes over the other devices, causing high latency and connections
disruption. I mean we are talking about connectivity, it should not happen.

~~~
slackingoff2017
Yup, it's because for the longest time TCP was as aggressive as possible to
get the most bandwidth.

Over the last 10 years networks buffers have become gigantic and sending
packets as fast as you can causes standing queues that can be a dozen seconds
long. This is where the lag comes from when someone on your LAN is
uploading/downloading.

The lag actually gets worse with higher connection speed because the devices
tend to have even bigger buffers.

BBL and to some extent VEGAS are smarter going about things. They attempt to
maintain constant latency instead of maximum bandwidth. For a few percent less
connection speed all the buffers stay empty and delay stays minimal

------
gtirloni
A nice article about kernel vs. userland network stacks at Cloudflare:
[https://blog.cloudflare.com/why-we-use-the-linux-kernels-
tcp...](https://blog.cloudflare.com/why-we-use-the-linux-kernels-tcp-stack/)

------
cromulen
As opposed to the fuchsia netstack written in Go by Google
[https://github.com/fuchsia-mirror/netstack](https://github.com/fuchsia-
mirror/netstack)

Why so many netstacks in Go? I would imagine even the tiny GC pauses Go has
would be undesirable for a netstack.

~~~
hugelgupf
Fuchsia's netstack is based off this netstack. Fuchsia's netstack is
essentially a wrapper of this netstack with link endpoint implementations to
access NICs via magenta.

------
omarforgotpwd
So i'm not an expert in kernel context switches... would running a network
stack in userland improve performance in any meaningful way for your average
network service written in go? or is there some other purpose for this?

~~~
justincormack
No, tcp is unlikely to be the bottleneck for a typical Go network server.

------
zbowling
It's the same netstack used in Fuchsia OS

------
ohnoesjmr
I wonder if one could abstract tcp on top of udp using this.

~~~
unmole
If by "abstract tcp", you mean implement reliability and in-order delivery
then, yes you can. QUIC[0] is the most promising of such approaches.

[0]: [https://en.wikipedia.org/wiki/QUIC](https://en.wikipedia.org/wiki/QUIC)

------
markonen
Not sure to what extent keeping up with the Joneses is a factor here (or
indeed which party is the one trying to keep up), but Apple recently moved to
a userspace/in-process TCP/IP stack in iOS 11 (and in the corresponding
watchOS and tvOS versions).

Seeing that this is for fuchsia makes that seem relevant.

------
slackingoff2017
Yes! I've been waiting for a fast userland OSS network stack forever. There's
things like DPDK for IP but nobody has taken up the monumental task of getting
TCP working and reliable

~~~
gonzo
f-stack, based on libuinet works over DPDK.
[https://github.com/F-Stack/f-stack](https://github.com/F-Stack/f-stack)

Libuinet is the FreeBSD TCP stack in user space (including over netmap)
[https://github.com/pkelsey/libuinet](https://github.com/pkelsey/libuinet)

There is also work in FD.io on userspace TCP & UDP.
[https://github.com/FDio/tldk](https://github.com/FDio/tldk)

------
oh_sigh
How does Google use this internally? Is it for cloud?

~~~
unmole
This seems to have been initially started as personal project by some
Googlers. Now it is used in Google's upcoming operating system Fuchsia:
[https://github.com/fuchsia-mirror/netstack](https://github.com/fuchsia-
mirror/netstack)

------
jwn
Can someone explain what the use cases for this are?

~~~
convolvatron
a few things come to mind :

    
    
      - unikernel go
      - making better parallel abstractions over network streams without having to deal with the mess of a kernel interface
      - as a platform to implement more integrated network policy (like congestion control, or state management for ddos protection)
      - ultimately might be really helpful for portability by requiring only a raw packet interface from the host OS
      - much easier environment for tying in w/ SDN and QOS (which google seems to be pretty gung ho on)
    

there are probably a lot more. I looked a little at the repo, and they went
for the most readable version that leans very heavily on the rich go
runtime...which I think is a great call if you want to play around. messing
with the linux network stack is a bit fussy and painful, there is a lot of ..
stuff.. in there

