
“It's been great to see Wireguard mature over the years” - zx2c4
https://plus.google.com/+gregkroahhartman/posts/jD6N4BzToa3
======
zx2c4
Greg KH, some others, and I had a great session stepping through every line of
the codebase yesterday. Even though the codebase is quite small -- under 4k
LoC -- studying some of the related topics is kind of like a fractal...

Happy to answer any questions you all might have about WireGuard. We're in
#wireguard on Freenode, too, if you need help with things in real-ish time.
I'm also mailing out free stickers to anybody who wants some:
[https://lists.zx2c4.com/pipermail/wireguard/2017-May/001338....](https://lists.zx2c4.com/pipermail/wireguard/2017-May/001338.html)
\-- just email team@wireguard.com and I'll throw them in an envelope.

~~~
majewsky
Hi Jason, thanks for your work on Wireguard! I evaluated it just a few weeks
ago for my own usecase, but couldn't get it to work yet. Maybe you can tell me
what I'm missing.

My usecase is as follows: I have a number of systems without publicly-routable
IPs (i.e. behind NAT in different private networks) and two servers with
public IPs (let's call these "head1" and "head2"). I want to build an overlay
network between all of these servers, so that they can all reach each other.
Since any of the two head servers may be down for whatever reason and I may
not always have time to immediately investigate the problem, I want to have a
highly-available setup where, upon failure of one head node, traffic goes over
to the other head node without long disruption. I tried to give the clients a
configuration like this:

    
    
      [Interface]
      PrivateKey = ...
      ListenPort = 12345
    
      [Peer]
      PublicKey = ...
      Endpoint = head1:12345
      AllowedIPs = 0.0.0.0/0
    
      [Peer]
      PublicKey = ...
      Endpoint = head2:12345
      AllowedIPs = 0.0.0.0/0
    

But when I set up the wg0 interface with that configuration, it would discard
the AllowedIPs setting for head1 upon finding the head2 peer with the same
AllowedIPs value (at least according to `wg`). Am I doing something wrong, or
is this an inherent limitation of Wireguard? (The fineprint: I did this two
weeks or so ago and am recalling this from memory right now, so I may be wrong
in the details.)

~~~
zx2c4
WireGuard enforces a very strict mapping between public keys and IP addresses,
so you can't have two public keys with the same internal tunnelled IP address
("AllowedIPs"). This way you can have rules like "if it comes from the
WireGuard interface and it's from 10.8.3.9, then it must be from public key
ABCD," which make firewalling and ACLs and the general security posture a lot
more straightforward.

In your case, it sounds like you want to change the AllowedIPs from one peer
to the other based on some kind of metric you have for determining disruption.
In this case, you can do that switch at runtime:

    
    
        $ wg set wg0 peer ABCD allowed-ips 0.0.0.0/0
    

or

    
    
        $ wg set wg0 peer EFG9 allowed-ips 0.0.0.0/0
    

Alternatively, if you wish to determine this disruption situation using
traditional routing daemons, that set actual routes, as opposed to WireGuard
AllowedIPs, then you could just have two separate WireGuard interfaces, and
leave it to your routing daemon to decide which one to use.

~~~
majewsky
The latter suggestion of using two wg interfaces sounds reasonable. I'll give
it a shot, thanks!

------
pedrocr
Seems more similar to tinc[1] than to OpenVPN. How well does it overcome
firewalls?

I've gotten used to just setup tinc on all my machines automatically with
puppet and it's really nice to have an overlay VPN over the internet that is
peer-to-peer instead of having to route everything through a central server.
The only thing I miss is better firewall piercing. The easiest way is to just
move tinc to 443 but that won't work for servers where I actually want to
serve HTTPS.

[1] [http://www.tinc-vpn.org/](http://www.tinc-vpn.org/)

~~~
yjftsjthsd-h
Question: I've looked at tinc and really want to use it, but I have
reservations about running a daemon as root on publicly accessible ports
without being _very_ confident in its security. I couldn't seem to find any
mention of them ever being audited, although [http://www.tinc-
vpn.org/goals/](http://www.tinc-vpn.org/goals/) lists it as a goal. They do
have a distinct lack of CVEs, but I can't tell if that's secure code or nobody
looking. Any chance you (or another commenter) can make a case that it's safe
to run tinc?

~~~
pedrocr
I hadn't looked at my config in a while and you're absolutely right, running
as root is a big warning flag. Just fixed it with the tincd "\--user" flag.
Created a new tinc user that owns all of /etc/tinc and set "\--user=tinc" in
/etc/default/tinc. Runs fine.

------
locusm
Ive been using Wireguard on Ubiquiti routers recently - its simplicity and
perf make it very attractive. Well done to the WG team.

------
phragma
I did a vuejs/html5 web interface for wireguard recently and found it
delightful to work with.

There was a few pain points on the way, if anyone is interested in learning
from my mistakes.

\- there is no option to specify listener ip+port tuples

\- logical sub-interfaces work fine, unless they happen to emit a vlan tag,
then wireguard drops the traffic:

ping: sendmsg: Protocol not supported

\- almost every virtual ethernet device in the kernel calls
eth_hw_addr_random(dev); to ensure a proper link-layer address, seems to be
missing in wg

\- "allowed-ips" cannot be turned off; when a routing table entry with a
gateway address is used to specify exactly which peer some subnet should be
routed to, then one has to also manually maintain an "allowed-ips" setting
that contains a duplicate of the information in the routing table

\- when one "allowed-ips" setting is added to a peer, wg may silently remove
"allowed-ips" on another, unrelated peer. this caught me by surprise while
trying to connect 2 VRFs from site A to site B and C over a common tunnel
infrastructure, which may have been stretching things a bit

Enjoy wireguard!

~~~
zx2c4
> I did a vuejs/html5 web interface for wireguard recently and found it
> delightful to work with.

Sounds cool. Care to share? I'd love to see people's usage.

> \- there is no option to specify listener ip+port tuples

WireGuard listens on all interfaces, because packets themselves are
authenticated cryptographically. If a packet makes it to your system and the
authtag is validated, it doesn't matter through what interface it arrived;
it's legit. WireGuard also is able to reply using the destination address and
interface of the incoming packet as the source address and interface of the
outgoing packet, which is the other common usage of binding to particular IP
addresses.

> \- logical sub-interfaces work fine, unless they happen to emit a vlan tag

WireGuard is layer 3 -- IP -- not Ethernet, which is how I believe tunnels
should be made. Instead of vlans, just add more tunnels if you need.

> \- almost every virtual ethernet device in the kernel calls
> eth_hw_addr_random(dev);

WireGuard isn't Ethernet; it's IP. It doesn't use MAC addresses. It has public
keys instead.

> \- "allowed-ips" cannot be turned off;

Yes: the crypto key routing is the central idea of WireGuard. You establish a
strong binding between public key and internal tunneled IP address, and then
all your rules and policies on top of that suddenly become quite simple.

> \- when one "allowed-ips" setting is added to a peer, wg may silently remove
> "allowed-ips" on another, unrelated peer

Peers can't share internal tunneled IP addresses; otherwise, there wouldn't be
strong binding to public keys, nor efficiency of sending outbound packets. So
if you assign an already assigned one to another peer, the presumption is that
you want to _move_ the allowed IP. By always going with the newer future
demand, and not erroring and staying with the old one, we ensure forward
progress on your overall changes to policy. IOW, if you're changing things, we
want the state to look like what you're changing it to.

If you have any additional questions at this level (a great level, thanks for
those), a better place to discuss is probably #wireguard on Freenode, or the
mailing list --
[https://lists.zx2c4.com/mailman/listinfo/wireguard](https://lists.zx2c4.com/mailman/listinfo/wireguard)

~~~
phragma
Yeah for sure. Here are some screenshots:

[https://imgur.com/a/aT8TU](https://imgur.com/a/aT8TU)

There are additional settings on the various detail screens (not shown), such
as wg peers.

Vuejs turned out to be quite cool. If you plug in a new linecard (like an USB
Ethernet adapter for example), it pops up in the GUI immediately and is ready
to be named, configured, used etc. ;-)

 _> WireGuard listens on all interfaces [...]_ _> If a packet makes it to your
system and the authtag is validated, it doesn't matter through what interface
it arrived; it's legit._

Yeah but you try telling the person administering the combined
firewall/vpn/router box that his/her firewall rules are no longer in control
of what traffic is allowed in or not. If it happens to be destined to the
wireguard port, the rules are for naught.

I should probably explain that there's a small firewall in there too. It uses
nftables to compile user rules via eBPF to machine code. The way these things
normally work, as I understand it, is that forwarding is disabled during boot,
and only when the firewall policy has been successfully loaded is forwarding
enabled via a sysctl flag.

The firewall handles simple stuff like masquerading and destination NAT.
Advanced functionality happens by handing off datagrams from the firewall to
userspace or kernelspace daemons like wireguard.

Long story short (too late, sorry), you can't have stuff listening on
0.0.0.0/0 and ::/0 because it is unaffected by the forwarding flag and hence
the firewall rules are violated during boot (and potentially for an unbounded
time period if the policy fails to load for some reason).

Anyway, turned out it was really easy to fix by patching the wireguard source
code.

 _> WireGuard is layer 3 -- IP -- not Ethernet, which is how I believe tunnels
should be made._

That's a completely fair point.

I'm just talking from a user's perspective, where what pops up on one's box
after installation is in fact a virtual ethernet device, in kernel-speak.

When the device then drops L2 frames, it can be a bit unexpected to someone
unfamiliar with that specific philosophy.

Is there any recommended workaround if one wants routers to see each other on
the link layer? Some kind of encapsulation maybe?

 _> WireGuard isn't Ethernet; it's IP. It doesn't use MAC addresses. It has
public keys instead._

Yeah I guess the public key sort of is the link layer address in this case.

 _> a better place to discuss is probably #wireguard on Freenode, or the
mailing list_

I was tempted to ask why the wg device shows up with the POINTOPOINT flag,
when it is clearly a multipoint device, but I'll save it for the IRC channel.
;-)

~~~
zx2c4
> Yeah for sure. Here are some screenshots:

Looks real nice. Any plans to open source that?

> Yeah but you try telling the person administering the combined
> firewall/vpn/router box that his/her firewall rules are no longer in control
> of what traffic is allowed in or not.

No, this is not correct. All iptables/nftables/netfilter rules apply. My
comment was strictly related to the fact that it listens on 0.0.0.0, but
you're still in full control of all firewalling in every way.

> stuff listening on 0.0.0.0/0 and ::/0 is unaffected by the forwarding flag

Either I don't understand what you've written, or this is complete nonsense.
This doesn't correspond with anything real I can fathom. Probably it'd be
easiest to just poke us in #wireguard on Freenode, and we can try to figure
out together whatever behavior you're seeing. In addition to me being in there
to discuss WireGuard-related things, there are some nftables experts there
too, who will probably have insights into your firewall configuration.

> is in fact a virtual ethernet device, in kernel-speak.

It's actually not an ethernet device. In "kernel-speak" it's a net_device with
hard_header_len = addr_len = 0, instantiable via rtnl.

> When the device then drops L2 frames

All L3 devices reject L2 frames. There are quite a few L3 devices, in fact,
not just WireGuard.

~~~
phragma
_> Looks real nice._

Thanks!

 _> Any plans to open source that?_

Not opposed to it, but on the other hand I have no idea how to package it?

I ended up implementing daemons for eg. setting up all the interfaces and
routing on the box, because I ran into so many bugs in systemd-networkd that
in the end it was impossible to work around them all and I just implemented it
from scratch.

So installing the software pretty much takes over your system and turns it
into a vpn+firewall unit. ;-)

Not something most users would expect if they find an open source project and
install a .deb or something. So I'm not sure if it is advisable to open source
it.

 _> All iptables/nftables/netfilter rules apply._

Rules haven't loaded yet, as per previous reply.

 _> Either I don't understand what you've written, > or this is complete
nonsense._

The former, I'm sure ;-).

 _> Probably it'd be easiest to just poke us in #wireguard_

Yeah maybe.

 _> In "kernel-speak" it's a net_device with hard_header_len = addr_len = 0,
instantiable via rtnl._

OK.

I actually have an alternative implementation of the GUI lying around where it
does handle wg devices like a special device that you can only do IP on.

But it turned out to be a lot of reimplementation of everything (handling IP
addresses, routing, configuring interfaces, generating anti-spoofing rules,
applying policy, ...) that in the end it turned out to be an unworkable mess.

So right now the code handles wg interfaces like any other "bearer" interface,
in other words a bit-bucket you can pour stuff into and it turns up at a peer
port on some other box. Obviously that is a bit wrong as it cannot forward L2,
so that means there are some features that are unusable (in particular VLAN
tagging and VRFs).

(It's also possible that I have just been too eager when selecting wireguard
and should have found an L2-compatible VPN system instead. Not sure which one
fits the bill though.)

By the way, is there a video available of the talk/session you and GKH and
others had recently?

I'd love to learn about wireguard internals by following along a recording of
the session.

~~~
phragma
Quick follow-up.

If the premise "the peer public key is sort of the peer's lladdr" is correct,
here's an idea off my chest. (Beware that I am blissfully unaware of wireguard
internals, just brainstorming.)

If the peer pubkey identifies the port on the other end, just like a MAC
address would identify the NIC on the other end of a switched network;

Then would it be possible to give the kernel the peer pubkey as the lladdr
when it asks for a next-hop's lladdr?

If so, all the IP routing stuff could be moved out of wireguard and taken care
of by the kernel's IP router.

Without the internal IP routing function, L2 traffic could possibly be
forwarded.

If the lladdr field is not large enough, maybe allow the user to give to wg a
mapping between fx. a 6-byte locally assigned peer lladdr, and the 32-byte
peer pubkey. Or ask Linus to extend the field. ;-)

Could be hidden behind an "l2-mode" flag or something.

In my opinion, being able to forward L2 would help a lot towards building a
GUI in a sensible way, but yeah, just an idea.

I'll find an IRC client soon, promise.

~~~
phragma
Or maybe a sub-interface per peer:

    
    
       # ip link add dev wg0 type wireguard
       # wg set wg0 listen 0.0.0.0:51820 listen ::/0:51820 private-key /path/to/private-key
       
       # ip link add dev wg0-peer1 type wireguard-peer
       # wg set wg0-peer1 local-endpoint wg0 remote-endpoint 209.202.254.14:8172 public-key ABCDEF
    
    

Then routing via kernel:

    
    
       # ip route add 192.0.2.0/24 dev wg0-peer1
    
    

And/or layer something else on top:

    
    
       # ip link add link wg0-peer1 name peer1-vlan100 type vlan id 100
    
    

Perhaps enslave that to a VRF:

    
    
       # ip link set peer1-vlan100 vrf blue
    
    

The peering interfaces could also be assigned an MTU that matches the path to
each peer, if known:

    
    
       # ip link set wg0-peer1 mtu 1328

------
j_s
What are the pros/cons comparing Wireguard and ZeroTier?

Wireguard was recently mentioned as supported by the Mullvad and IVPN VPN
providers.
[https://news.ycombinator.com/item?id=15588040](https://news.ycombinator.com/item?id=15588040)

ZeroTier was part of a discussion migrating between cloud providers.
[https://news.ycombinator.com/item?id=15548642](https://news.ycombinator.com/item?id=15548642)

Choice is great but when would either option be best? My personal use is just
as a connect-back-through home VPN; I have to support mobile devices so will
probably also have to use something else.

Both tools seem to be finding new life as they simplify container networking
setups.

------
1_player
Either I'm dumb or the guide on
[https://www.wireguard.com](https://www.wireguard.com) is awful.

I have a VPS with StrongSwan installed, through which I connect when I need to
access the internet with my server's IP. Basically, a poor sysadmin's VPN.
I've heard of WireGuard, why not try it out?

Now, the installation page seems pretty easy to follow, and I guess (though
it's implied) WireGuard has to be installed on the server.

Then the Quick Start page shows a list of routing commands, where am I
supposed to run them? On the client or the server?

Or, quote:

    
    
        wg set wg0 listen-port 51820 private-key /path/to/private-key peer ABCDEF... allowed-ips 192.168.88.0/24 endpoint 209.202.254.14:8172
    

What are those IPs? What's that private key file?

How the heck do I install and use it?

Also, the side-by-side demo doesn't load on my Safari.

I'm fine RTFM and man pages, but I don't see the point of a quick start guide
if it's totally cryptic for anyone that has never used WireGuard. Just tell me
to RTFM and I won't bother with this guide.

I guess I'll keep my StrongSwan installation, has been working without a hitch
for a while now.

~~~
zx2c4
Thanks for the feedback. Indeed there are the wg(8) and wg-quick(8) man pages,
and probably you should read the whole _front page_ of wireguard.com, namely
this part -- [https://www.wireguard.com/#conceptual-
overview](https://www.wireguard.com/#conceptual-overview) \-- which gives some
important conceptual overviews to learn what's going on. Probably if you don't
have the conceptual overview in mind, the quickstart won't make much sense;
this may indeed be a fault in the writing presumptions of the quickstart.

The side-by-side video puts this -- [https://www.wireguard.com/talks/talk-
demo-screencast.mp4](https://www.wireguard.com/talks/talk-demo-screencast.mp4)
\-- in a <video> tag. I'm a kernel developer, not a web developer, so if you
have a suggestion on how to fix it in your browser, I'm all ears.

With all that said, we probably should rewrite the quickstart page to be a
little more intuitive and "quicker", so that people can get an idea of what to
do without having to understand or think about anything... Maybe a better
approach for that might be: "Here's what you do on one side. Here's what you
do on the other side."

In any case, I guarantee that learning WireGuard is faster and simpler than
learning StrongSwan or any IPsec things.

After the Netdev conference, which is next week, I'll allocate some time to
improving documentation. Thanks for the feedback here, and sorry for the
frustration.

~~~
1_player
BTW, Firefox says the video is corrupt, Safari doesn't play it, only Chrome
does. Probably a couple bits rotted since you've recorded it :)

~~~
zx2c4
Jeepers creepers. I'll try to fix it. Probably a yuv444p issue.

Update: fixed now. I had to re-encode it to change the colorspace, as
suspected, and now things should be working fine in Firefox (though I haven't
a Mac to test Safari).

~~~
zie
It works on Safari 11.0 as well(source: me). Thanks for fixing it!

------
binaryapparatus
Mixed feelings, some of them maybe unpopular:

1\. Careful with anything that Greg KH prizes.

2\. Written by zx2c4 who is very high on my 'who I can trust' list.

Since #2 greatly over-weights #1 can't wait to try WireGuard.

~~~
mediocrejoker
Can you elaborate on #1?

~~~
binaryapparatus
Exactly as other post says, kdbus and the way he tried to push it raised a few
warning flags. Kernel is all about trust and I just don't trust Greg after
that episode.

It is all off topic since I really admire zx2c4 and this is about his product.

Edit: since Greg KH got removed from the thread title any mention of him is
off topic...

~~~
j_s
FWIW, it is his Google+ account in the URL. Definitely no longer obvious.

Hacker News Transparency:
[https://hn.0x2237.club/post/15596963](https://hn.0x2237.club/post/15596963)

------
rconti
Context?

For those of us who have never heard of Wireguard, I can click on the linked
Wireguard image to at least see what it is. Cool, I guess? If it was a
standalone link on HN ("hey, look what we made!") it would make a lot more
sense, but since it's a G+ post, it's more about the blog-type content. I
guess the main point is the discussion of merging it into the (Linux?) kernel
tree, but... yeah.

~~~
majewsky
Wireguard by itself has been on HN for a few times now. The newsworthy part is
that one of the top kernel developers is expressing interest in bringing it
into mainline, so it is appropriate to submit that post instead of the
Wireguard homepage.

~~~
rconti
Thanks! I didn't realize who Greg was either, so I was missing the 2 pieces of
context necessary for the post to make any sense.

