
BoringTun, a Userspace WireGuard Implementation in Rust - jgrahamc
https://blog.cloudflare.com/boringtun-userspace-wireguard-rust/
======
FredFS456
Jason also sent an email to the WireGuard mailing list addressing this. [1] In
the email, he stated that Cloudflare repeated refused to work directly with
him and his team as part of the upstream project.

[1]
[https://lists.zx2c4.com/pipermail/wireguard/2019-March/00404...](https://lists.zx2c4.com/pipermail/wireguard/2019-March/004048.html)

~~~
CodeWriter23
One of CloudFlare's goals is to have a BSD-Licensed result. Seems reasonable
to maintain isolation from the core WireGuard team that is publishing under
GPL to support that goal, just to ensure they don't accidentally end up with
GPL code in their codebase.

~~~
zx2c4
With the exception of the kernel-specific code, we're actually pushing more
and more of our stuff as MIT/Apache2 licensed. If there's a project we haven't
done that to yet that's not permissively licensed, just email team at
wireguard dot com or ask on the mailing list. wireguard-go, for example, is
MIT.

~~~
xucheng
It seems that the official rust implementation wireguard-rs[1] is currently
GPL licensed.

[1]: [https://github.com/WireGuard/wireguard-
rs](https://github.com/WireGuard/wireguard-rs)

------
gorset
Nice to see more userspace implementations of wireguard.

I've been testing out wireguard's linux kernel module, but experienced too
much jitter due to wireguard not respecting isolcpus.

Probably possible to make the kernel module behave, but working with userspace
is so much easier. Wireguard-go seems to behave (the drop in performance is
not necessarily a showstopper), although the description says that it
shouldn't be used on Linux :-)

Would be nice if a high performance rust implementation can become a real
alternative. A userspace implementation shouldn't need to be much slower than
a kernel module if network card with kernel bypass is available, if you really
need that extra performance.

------
atonse
Unfortunate that Cloudflare didn't just contribute time to the first party
wireguard-rs client. But it's their wish to do what they want with their
engineers.

Nonetheless, it's great that Wireguard have a path to a solid user space
implementation that others could build from.

Jason Donenfeld's post about this surprised me when he said the go
implementation is flaky. Is the health of the non-kernel parts of wireguard
not great? The macOS clients work really well for me.

~~~
zx2c4
The Go implementation is fine. I don't think you have much to worry about.

But we're pretty frequently running into limitations and fighting with the
runtime; we've submitted quite a few random patches to it. I think Rust is
probably a better-suited language for a high performance and relatively secure
WireGuard implementation. Hence, we'd really like to have a strong first-party
Rust implementation for the project.

~~~
BFLpL0QNek
Haven't looked at the other implementations yet but with wireguard-go I've
been struggling to run it as non root on OpenBSD.

I've created a _wireguard user+group, setup a /dev/tun0 interface to have the
_wireguard group with read/write however from the little tracing i've done it
looks like I still need root when it tries to do the unix sys call to set the
MTU.

------
zoobab
OpenVPN is a piece of shit performance wise, the best userspace VPN tool I
tested was curvetun. It was moving encrypted gigabit speeds with very low CPU
usage on MIPS openwrt routers.

[http://netsniff-ng.org/](http://netsniff-ng.org/)

------
geogriffin
Looks like pretty good code, but interesting that they only have a few
dependencies and decided to implement a lot of stuff, like kernel polling,
themselves rather than using the mio crate. I wonder why.

They also implemented the noise handshake in a non-generic way tightly coupled
to wireguard protocol, again written themselves instead of using an external
dependency, which I think is also the approach other wireguard implementations
have taken.. which is fine, but harder to independently verify. There's a lot
of code there.

~~~
thecompilr
I have been getting the mio question a lot.

Mio, in theory, is not very far off of what I ended up implementing, but it is
missing a few features that I really needed, and building those features on
top of Mio would be harder than just using epoll/kqueue directly. Also I don't
understand the fear people have of using epoll/kqueue. Those are very basic
features that every engineer has to know how to use.

So what are the features:

No locks - Mio polls under lock, but epoll/kqueue are multithread safe by
nature as long as you use the correct filters. Mio has considerable overhead
over pure epoll/kqueue.

No need for tokens, instead I store closure pointers. Real zero overhead. Of
course I could cast pointers to usize and use as Mio tokens, but I would still
have to build the ownership model on top, which is most of the work anyway.

Timers in Mio run in a thread??? I use platform idiomatic conventions instead,
which with kqueue require direct access to kqueue.

Signal handlers, can't make idiomatic one with Mio (or maybe I missed how).

Same with notifications. Could use something awkward like a channel, but
prefer idiomatic approach.

------
gok
Nice, Rust is a great fit for this kind of project.

------
LinuxBender
What does the context switching overhead look like in this implementation, vs
kernel space? Are there any throughput graphs comparing the two?

~~~
thecompilr
It is still quite expensive, the kernel module is still faster. Performance
depends a lot on your kernel version and network card. We measured anywhere
from 20% to 40% advantage to kernel module. But we are working on additional
performance features, such as using send/recvmmsg and raw sockets.

~~~
LinuxBender
Thankyou, that is even better than what I would expect. I have been trying to
tune around similar issues in tinc-vpn which also runs in user space and
depends on newer kernels and code changes to work around.

------
rapsey
Blog post mentions windows support but there is no real code support for it.
Will it eventually be ported?

~~~
thecompilr
Currently the library part supports Windows, and it can be used from C# or
C++, if you want to use something like [https://docs.microsoft.com/en-
us/uwp/api/Windows.Networking....](https://docs.microsoft.com/en-
us/uwp/api/Windows.Networking.Vpn)

However there is no native support for tunnel interfaces in Windows, and we
are still thinking about the best way to add full support.

The WireGuard project for example developed a whole new driver just for that.
Don't know if we are going to follow that route.

~~~
rapsey
They did and the code is surprisingly short at only ~1300 lines of C. Why not
just use their driver?

------
AceJohnny2
I've been interested in using WireGuard for a while, though I don't really
have a use-case for it currently.

In the interest of honest understanding, what would be reasons _not_ to use
WireGuard over existing VPN software (excluding the obvious legacy interop) ?

~~~
atonse
I am a huge fan of Wireguard and use on all the major production servers I
currently manage.

But user management is exactly as sophisticated as SSH key management. That is
to say, easy for 2-3 people, a pain otherwise.

There are attempts at creating UI based user managers for wireguard, and I
hope those go somewhere.

I bet the major wireguard-supporting VPN providers have solved this problem
for themselves though.

Update: Just wanted to add that I don't think that user management (the work
of actually provisioning and managing users) should fall under the umbrella of
Wireguard. It's perfect the way it is. But I suspect there will be an
ecosystem of tools that come up around it.

~~~
AceJohnny2
Thanks, this is exactly the kind of useful information I was looking for.

(this wouldn't be an issue for my personal use, but it's useful to be aware of
it)

~~~
jandrese
It's not an issue for like 95% of the users, which is why Wireguard went that
way. For the people who really need something better they are left to figure
it out for themselves. Normally this is OK, but sometimes it leads to
situations like PGP Email where key distribution is never really solved and
the whole thing kind of lumbers around in a half dead state forever.

------
merb
is really wireguard that better than other VPNs? I always wondered if there is
need for a "better" vpn.

I mean a more reliable VPN is probably needed, i.e. something that isn't
detectable. However I guess wireguard has all the problems than most VPNs once
it's known how it works it's detectable with a DPI. What I probably want to
see is a VPN over h2c or over quick, its probably impossible to detect it, it
looks like regular web traffic.

~~~
SEJeff
Wireguard focuses on being a better vpn where better is vaguely defined as
using modern crypto in a way that is hard to get wrong. As a result, it is a
lot simpler with far fewer bells and whistles. This means it tends to be
orders of magnitude faster than an equivalent usage of say openvpn (I measured
15mbps vs 52mbps awhile back with openvpn and wireguard respectively).

What you want is a slightly different albeit very important problem to solve
reliably.

------
zoobab
Any Dockerfile?

~~~
zamadatix
Open Issue
[https://github.com/cloudflare/boringtun/issues/38](https://github.com/cloudflare/boringtun/issues/38)

------
jedisct1
Is the name resemblance with GloryTun (another VPN) just pure coincidence?

~~~
zamadatix
According to the blog post:

"The name might sound a bit boring but there's a reason for it: BoringTun
creates a tunnel by 'boring' it. And it’s a nod to Google’s BoringSSL which
strips the complexity out of OpenSSL. We think WireGuard has the opportunity
to do the same for legacy VPN protocols like OpenVPN. And we hope BoringTun
can be a valuable tool as part of that ecosystem."

so coincidental.

