Greg KH, some others, and I had a great session stepping through every line of the codebase yesterday. Even though the codebase is quite small -- under 4k LoC -- studying some of the related topics is kind of like a fractal...
Happy to answer any questions you all might have about WireGuard. We're in #wireguard on Freenode, too, if you need help with things in real-ish time. I'm also mailing out free stickers to anybody who wants some: https://lists.zx2c4.com/pipermail/wireguard/2017-May/001338.... -- just email team@wireguard.com and I'll throw them in an envelope.
First of all, love the project. Thanks for tackling it and sticking to it.
I know that talking about a windows client is really early at this point, but is your team considering implementing it in a way that actually uses the UWP VPN provider interfaces (https://docs.microsoft.com/en-us/uwp/api/Windows.Networking....) instead of (or in addition to) the OpenVPN-style system tray app? I know there's been some strides towards using UWP from rust. And if you're thinking about it, early consideration might give extra time for any necessary interactions with MS.
It would be really nice to have an open-source VPN implementation actually integrate with the OS, and I think it might provide the impetus people need to ditch OpenVPN. So even if you don't plan on directly working to integrate with the OS interfaces, please consider referencing the APIs when working on userspace libraries. Keeping the needs in mind could at least simplify any future efforts in that regard.
The OpenVPN Windows kernel TUN/TAP driver is really super scary. That alone has a larger code base than all of WireGuard...
At the moment, the efforts around the cross-platform userspace-based WireGuard implementations have focused on targeting that TUN/TAP driver, which is pretty ugly business. But thanks very much for pointing me toward this more general purpose API.
It might wind up being _easier_ to use this than having to talk to terrible OpenVPN kernel drivers. I'll investigate this thoroughly.
By the way, if you're into Windows programming and want to help out, don't hesitate to email team@wireguard.com.
> The OpenVPN Windows kernel TUN/TAP driver is really super scary.
I believe you. Everything about OpenVPN code scares me. Not sure if it helps, but I checked out an article by Dmitri Varsanofiev on using the TUN/TAP driver (http://www.varsanofiev.com/inside/using_tuntap_under_windows...) and it seems he was able to work with the driver from managed (C#) code without actually modifying the driver source at all. It seems he was just invoking two Win32 API call methods from Kernel32.dll: CreateFile and DeviceIoControl. If he can do it from C#, it should be doable from rust.
From his sample, it seems you could work with the tunnel device without modifying the source code at all. Although that means you've still got a major code wart from OpenVPN haunting you. And even if it works, I'm not sure if that's the recommended approach. But if you're insistent on supporting Wireguard on Windows versions before 10, the TUN/TAP driver might be the only route available. I don't think those VPN APIs existed in Windows 7 and I think they were private in 8/8.1, if memory serves.
The good news about the UWP APIs is that you once you confirm that it works, you could potentially get it up on the Windows store and make it easy for people to install and update. Though there's tons of notes that the VPN APIs are restricted, and you can only publish apps that use them after you get reviewed and your account gets those permissions. You can still sideload, (locally install) though. Honestly, I don't think the review would be a huge obstacle, though. I think Wireguard has some fans in MS.
> By the way, if you're into Windows programming and want to help out, don't hesitate to email team@wireguard.com.
I mostly work on the managed side of things, but I try to keep fresh on APIs and platform features. I'm not sure my workload really allows me to be a major part of the project, but if you've got a public portal that tracks your tasks, I might be able to subscribe and pop in and do some legwork from time to time. Especially if someone's writing the core functionality in some DLL that's easy to call. Do you have a public issue tracker? Or is it mostly IRC and mailing lists?
> he was able to work with the driver from managed (C#) code without actually modifying the driver source at all
Yea, I wouldn't need to (or want to) modify the source, and using the actual driver from userspace isn't that bad. Also, having a pre-signed win32k driver is a nice thing that I wouldn't want to give up. My motivation is mainly to avoid using it all together, if possible, which is I think what the API you linked to prior will allow.
So the UWP API looks like the right way forward. I wonder if we could launch it as Win 10+ without too many complaints. Nobody is really using 8/8.1, but there are of course corporate Win7 holdouts.
> Do you have a public issue tracker? Or is it mostly IRC and mailing lists?
There's wireguard.com/todo/ but it hasn't been populated with the particular TODOs for the userspace stuff. But if you send me an email or come into the IRC, I can fill you in at the optimal time when it seems like _the one thing remaining_ is just to fill in some platform-specific stubs for win32. If that's appealing to you, of course.
Hi Jason, thanks for your work on Wireguard! I evaluated it just a few weeks ago for my own usecase, but couldn't get it to work yet. Maybe you can tell me what I'm missing.
My usecase is as follows: I have a number of systems without publicly-routable IPs (i.e. behind NAT in different private networks) and two servers with public IPs (let's call these "head1" and "head2"). I want to build an overlay network between all of these servers, so that they can all reach each other. Since any of the two head servers may be down for whatever reason and I may not always have time to immediately investigate the problem, I want to have a highly-available setup where, upon failure of one head node, traffic goes over to the other head node without long disruption. I tried to give the clients a configuration like this:
But when I set up the wg0 interface with that configuration, it would discard the AllowedIPs setting for head1 upon finding the head2 peer with the same AllowedIPs value (at least according to `wg`). Am I doing something wrong, or is this an inherent limitation of Wireguard? (The fineprint: I did this two weeks or so ago and am recalling this from memory right now, so I may be wrong in the details.)
WireGuard enforces a very strict mapping between public keys and IP addresses, so you can't have two public keys with the same internal tunnelled IP address ("AllowedIPs"). This way you can have rules like "if it comes from the WireGuard interface and it's from 10.8.3.9, then it must be from public key ABCD," which make firewalling and ACLs and the general security posture a lot more straightforward.
In your case, it sounds like you want to change the AllowedIPs from one peer to the other based on some kind of metric you have for determining disruption. In this case, you can do that switch at runtime:
$ wg set wg0 peer ABCD allowed-ips 0.0.0.0/0
or
$ wg set wg0 peer EFG9 allowed-ips 0.0.0.0/0
Alternatively, if you wish to determine this disruption situation using traditional routing daemons, that set actual routes, as opposed to WireGuard AllowedIPs, then you could just have two separate WireGuard interfaces, and leave it to your routing daemon to decide which one to use.
You can't have overlapping AllowedIPs for peers on the same interface. Wireguard has to decide which encryption/decryption keys to use based on the AllowedIPs.
For your use case you need two wg Interfaces on each host (w/ different ListenPort and AllowedIPs=0.0.0.0/0) and a routing daemon (like bird).
So wireguard is focused on the use case of having a single central point where all the traffic has to go through? Reading throught the page it seemed it was more the tinc model of distributed P2P VPN overlay which to me seems much more powerful. Redundancy comes for free and performance is improved because you get direct links between nodes instead of having to proxy all the traffic.
Porting to FreeBSD, where new GPL2 code in base is verboten. Whether you consider FreeBSD non-free or not is a matter of opinion.
Even if the existing kernel implementation is highly tied to Linux, it is legally problematic for FreeBSD developers to reference GPL2 code while writing BSD-licensed code.
I think for FreeBSD kernel-land, there will have to be significant rewrites, anyway, since the Linux code is fairly Linux-specific. At some point though, I'd love to do a kernel implementation for the BSDs (in addition to the userspace one, which is coming much sooner), and I suppose you could expect for that to be BSD-licensed, appropriately.
> I think for FreeBSD kernel-land, there will have to be significant rewrites, anyway, since the Linux code is fairly Linux-specific.
Probably true!
> At some point though, I'd love to do a kernel implementation for the BSDs
If BSD developers can reference the Linux implementation's source, it is easier for them to go ahead and do it independently :-). Do you anticipate having time to do a BSD kernel implementation soon yourself, or is it just a would-be-nice idea on the infinite TODO list? I'm trying to gauge whether we should wait for you or try and do it ourselves.
It might not be about commercializing so much as including it in some of the *BSDs which don't allow for GPL code in the core. Otherwise, they'll probably have to write a new implementation. But yes, there is that risk.
Seems more similar to tinc[1] than to OpenVPN. How well does it overcome firewalls?
I've gotten used to just setup tinc on all my machines automatically with puppet and it's really nice to have an overlay VPN over the internet that is peer-to-peer instead of having to route everything through a central server. The only thing I miss is better firewall piercing. The easiest way is to just move tinc to 443 but that won't work for servers where I actually want to serve HTTPS.
Have you tried ipop/social-vpn[0]? It uses libjingle (stun, turn, etc) to get through firewalls. I don't know much about it's security though, or if it has been audited.
I looked at tinc yesterday, but it seems there are two options:
1. the stable release which only supports RSA-2048 keys (giving it RSA-4096 keys resulted in really bizarre errors like "read public key file failed: Operation now in progress"), and
2. the unstable release which does not compile on my system because of broken dependencies (and even if I got it to work, I have huge reservations about using unstable releases of cryptographic systems).
It's a small open source project that works very well for a lot of people. If you need stronger keys for your use case you'll have to wait for 1.1 to be stable. For me VPNs are not about security they're about getting connectivity working between my machines in a uniform way on top of networks that have restrictions. For security I always assume the network is hostile, even if it is my own LAN or VPN.
Question: I've looked at tinc and really want to use it, but I have reservations about running a daemon as root on publicly accessible ports without being very confident in its security. I couldn't seem to find any mention of them ever being audited, although http://www.tinc-vpn.org/goals/ lists it as a goal. They do have a distinct lack of CVEs, but I can't tell if that's secure code or nobody looking. Any chance you (or another commenter) can make a case that it's safe to run tinc?
I hadn't looked at my config in a while and you're absolutely right, running as root is a big warning flag. Just fixed it with the tincd "--user" flag. Created a new tinc user that owns all of /etc/tinc and set "--user=tinc" in /etc/default/tinc. Runs fine.
I don't believe this is something wireguard cares about. It looks like they're aiming for doing in-kernel build-your-own routing VPN solution for servers where the networking is completely within your control. If you need a solution for privacy / working around provider limitations, then other software may be better.
I did a vuejs/html5 web interface for wireguard recently and found it delightful to work with.
There was a few pain points on the way, if anyone is interested in learning from my mistakes.
- there is no option to specify listener ip+port tuples
- logical sub-interfaces work fine, unless they happen to emit a vlan tag, then wireguard drops the traffic:
ping: sendmsg: Protocol not supported
- almost every virtual ethernet device in the kernel calls eth_hw_addr_random(dev); to ensure a proper link-layer address, seems to be missing in wg
- "allowed-ips" cannot be turned off; when a routing table entry with a gateway address is used to specify exactly which peer some subnet should be routed to, then one has to also manually maintain an "allowed-ips" setting that contains a duplicate of the information in the routing table
- when one "allowed-ips" setting is added to a peer, wg may silently remove "allowed-ips" on another, unrelated peer. this caught me by surprise while trying to connect 2 VRFs from site A to site B and C over a common tunnel infrastructure, which may have been stretching things a bit
> I did a vuejs/html5 web interface for wireguard recently and found it delightful to work with.
Sounds cool. Care to share? I'd love to see people's usage.
> - there is no option to specify listener ip+port tuples
WireGuard listens on all interfaces, because packets themselves are authenticated cryptographically. If a packet makes it to your system and the authtag is validated, it doesn't matter through what interface it arrived; it's legit. WireGuard also is able to reply using the destination address and interface of the incoming packet as the source address and interface of the outgoing packet, which is the other common usage of binding to particular IP addresses.
> - logical sub-interfaces work fine, unless they happen to emit a vlan tag
WireGuard is layer 3 -- IP -- not Ethernet, which is how I believe tunnels should be made. Instead of vlans, just add more tunnels if you need.
> - almost every virtual ethernet device in the kernel calls eth_hw_addr_random(dev);
WireGuard isn't Ethernet; it's IP. It doesn't use MAC addresses. It has public keys instead.
> - "allowed-ips" cannot be turned off;
Yes: the crypto key routing is the central idea of WireGuard. You establish a strong binding between public key and internal tunneled IP address, and then all your rules and policies on top of that suddenly become quite simple.
> - when one "allowed-ips" setting is added to a peer, wg may silently remove "allowed-ips" on another, unrelated peer
Peers can't share internal tunneled IP addresses; otherwise, there wouldn't be strong binding to public keys, nor efficiency of sending outbound packets. So if you assign an already assigned one to another peer, the presumption is that you want to _move_ the allowed IP. By always going with the newer future demand, and not erroring and staying with the old one, we ensure forward progress on your overall changes to policy. IOW, if you're changing things, we want the state to look like what you're changing it to.
If you have any additional questions at this level (a great level, thanks for those), a better place to discuss is probably #wireguard on Freenode, or the mailing list -- https://lists.zx2c4.com/mailman/listinfo/wireguard
There are additional settings on the various detail screens (not shown), such as wg peers.
Vuejs turned out to be quite cool. If you plug in a new linecard (like an USB Ethernet adapter for example), it pops up in the GUI immediately and is ready to be named, configured, used etc. ;-)
> WireGuard listens on all interfaces [...]> If a packet makes it to your system and the authtag is validated, it doesn't matter through what interface it arrived; it's legit.
Yeah but you try telling the person administering the combined firewall/vpn/router box that his/her firewall rules are no longer in control of what traffic is allowed in or not. If it happens to be destined to the wireguard port, the rules are for naught.
I should probably explain that there's a small firewall in there too. It uses nftables to compile user rules via eBPF to machine code. The way these things normally work, as I understand it, is that forwarding is disabled during boot, and only when the firewall policy has been successfully loaded is forwarding enabled via a sysctl flag.
The firewall handles simple stuff like masquerading and destination NAT. Advanced functionality happens by handing off datagrams from the firewall to userspace or kernelspace daemons like wireguard.
Long story short (too late, sorry), you can't have stuff listening on 0.0.0.0/0 and ::/0 because it is unaffected by the forwarding flag and hence the firewall rules are violated during boot (and potentially for an unbounded time period if the policy fails to load for some reason).
Anyway, turned out it was really easy to fix by patching the wireguard source code.
> WireGuard is layer 3 -- IP -- not Ethernet, which is how I believe tunnels should be made.
That's a completely fair point.
I'm just talking from a user's perspective, where what pops up on one's box after installation is in fact a virtual ethernet device, in kernel-speak.
When the device then drops L2 frames, it can be a bit unexpected to someone unfamiliar with that specific philosophy.
Is there any recommended workaround if one wants routers to see each other on the link layer? Some kind of encapsulation maybe?
> WireGuard isn't Ethernet; it's IP. It doesn't use MAC addresses. It has public keys instead.
Yeah I guess the public key sort of is the link layer address in this case.
> a better place to discuss is probably #wireguard on Freenode, or the mailing list
I was tempted to ask why the wg device shows up with the POINTOPOINT flag, when it is clearly a multipoint device, but I'll save it for the IRC channel. ;-)
> Yeah but you try telling the person administering the combined firewall/vpn/router box that his/her firewall rules are no longer in control of what traffic is allowed in or not.
No, this is not correct. All iptables/nftables/netfilter rules apply. My comment was strictly related to the fact that it listens on 0.0.0.0, but you're still in full control of all firewalling in every way.
> stuff listening on 0.0.0.0/0 and ::/0 is unaffected by the forwarding flag
Either I don't understand what you've written, or this is complete nonsense. This doesn't correspond with anything real I can fathom. Probably it'd be easiest to just poke us in #wireguard on Freenode, and we can try to figure out together whatever behavior you're seeing. In addition to me being in there to discuss WireGuard-related things, there are some nftables experts there too, who will probably have insights into your firewall configuration.
> is in fact a virtual ethernet device, in kernel-speak.
It's actually not an ethernet device. In "kernel-speak" it's a net_device with hard_header_len = addr_len = 0, instantiable via rtnl.
> When the device then drops L2 frames
All L3 devices reject L2 frames. There are quite a few L3 devices, in fact, not just WireGuard.
Not opposed to it, but on the other hand I have no idea how to package it?
I ended up implementing daemons for eg. setting up all the interfaces and routing on the box, because I ran into so many bugs in systemd-networkd that in the end it was impossible to work around them all and I just implemented it from scratch.
So installing the software pretty much takes over your system and turns it into a vpn+firewall unit. ;-)
Not something most users would expect if they find an open source project and install a .deb or something. So I'm not sure if it is advisable to open source it.
> All iptables/nftables/netfilter rules apply.
Rules haven't loaded yet, as per previous reply.
> Either I don't understand what you've written,
> or this is complete nonsense.
The former, I'm sure ;-).
> Probably it'd be easiest to just poke us in #wireguard
Yeah maybe.
> In "kernel-speak" it's a net_device with hard_header_len = addr_len = 0, instantiable via rtnl.
OK.
I actually have an alternative implementation of the GUI lying around where it does handle wg devices like a special device that you can only do IP on.
But it turned out to be a lot of reimplementation of everything (handling IP addresses, routing, configuring interfaces, generating anti-spoofing rules, applying policy, ...) that in the end it turned out to be an unworkable mess.
So right now the code handles wg interfaces like any other "bearer" interface, in other words a bit-bucket you can pour stuff into and it turns up at a peer port on some other box. Obviously that is a bit wrong as it cannot forward L2, so that means there are some features that are unusable (in particular VLAN tagging and VRFs).
(It's also possible that I have just been too eager when selecting wireguard and should have found an L2-compatible VPN system instead. Not sure which one fits the bill though.)
By the way, is there a video available of the talk/session you and GKH and others had recently?
I'd love to learn about wireguard internals by following along a recording of the session.
If the premise "the peer public key is sort of the peer's lladdr" is correct, here's an idea off my chest. (Beware that I am blissfully unaware of wireguard internals, just brainstorming.)
If the peer pubkey identifies the port on the other end, just like a MAC address would identify the NIC on the other end of a switched network;
Then would it be possible to give the kernel the peer pubkey as the lladdr when it asks for a next-hop's lladdr?
If so, all the IP routing stuff could be moved out of wireguard and taken care of by the kernel's IP router.
Without the internal IP routing function, L2 traffic could possibly be forwarded.
If the lladdr field is not large enough, maybe allow the user to give to wg a mapping between fx. a 6-byte locally assigned peer lladdr, and the 32-byte peer pubkey. Or ask Linus to extend the field. ;-)
Could be hidden behind an "l2-mode" flag or something.
In my opinion, being able to forward L2 would help a lot towards building a GUI in a sensible way, but yeah, just an idea.
# ip link add dev wg0 type wireguard
# wg set wg0 listen 0.0.0.0:51820 listen ::/0:51820 private-key /path/to/private-key
# ip link add dev wg0-peer1 type wireguard-peer
# wg set wg0-peer1 local-endpoint wg0 remote-endpoint 209.202.254.14:8172 public-key ABCDEF
Then routing via kernel:
# ip route add 192.0.2.0/24 dev wg0-peer1
And/or layer something else on top:
# ip link add link wg0-peer1 name peer1-vlan100 type vlan id 100
Perhaps enslave that to a VRF:
# ip link set peer1-vlan100 vrf blue
The peering interfaces could also be assigned an MTU that matches the path to each peer, if known:
Choice is great but when would either option be best? My personal use is just as a connect-back-through home VPN; I have to support mobile devices so will probably also have to use something else.
Both tools seem to be finding new life as they simplify container networking setups.
I have a VPS with StrongSwan installed, through which I connect when I need to access the internet with my server's IP. Basically, a poor sysadmin's VPN. I've heard of WireGuard, why not try it out?
Now, the installation page seems pretty easy to follow, and I guess (though it's implied) WireGuard has to be installed on the server.
Then the Quick Start page shows a list of routing commands, where am I supposed to run them? On the client or the server?
Or, quote:
wg set wg0 listen-port 51820 private-key /path/to/private-key peer ABCDEF... allowed-ips 192.168.88.0/24 endpoint 209.202.254.14:8172
What are those IPs? What's that private key file?
How the heck do I install and use it?
Also, the side-by-side demo doesn't load on my Safari.
I'm fine RTFM and man pages, but I don't see the point of a quick start guide if it's totally cryptic for anyone that has never used WireGuard. Just tell me to RTFM and I won't bother with this guide.
I guess I'll keep my StrongSwan installation, has been working without a hitch for a while now.
Thanks for the feedback. Indeed there are the wg(8) and wg-quick(8) man pages, and probably you should read the whole _front page_ of wireguard.com, namely this part -- https://www.wireguard.com/#conceptual-overview -- which gives some important conceptual overviews to learn what's going on. Probably if you don't have the conceptual overview in mind, the quickstart won't make much sense; this may indeed be a fault in the writing presumptions of the quickstart.
The side-by-side video puts this -- https://www.wireguard.com/talks/talk-demo-screencast.mp4 -- in a <video> tag. I'm a kernel developer, not a web developer, so if you have a suggestion on how to fix it in your browser, I'm all ears.
With all that said, we probably should rewrite the quickstart page to be a little more intuitive and "quicker", so that people can get an idea of what to do without having to understand or think about anything... Maybe a better approach for that might be: "Here's what you do on one side. Here's what you do on the other side."
In any case, I guarantee that learning WireGuard is faster and simpler than learning StrongSwan or any IPsec things.
After the Netdev conference, which is next week, I'll allocate some time to improving documentation. Thanks for the feedback here, and sorry for the frustration.
Jeepers creepers. I'll try to fix it. Probably a yuv444p issue.
Update: fixed now. I had to re-encode it to change the colorspace, as suspected, and now things should be working fine in Firefox (though I haven't a Mac to test Safari).
Thanks, I hope I've given you some constructive criticism and wasn't too harsh.
I'm really interested, I hoped it was something I could install quickly and easily at 10pm before bed, I guess I'll study it better during the weekend and compare it against StrongSwan.
I struggled with understanding it, too. Even after reading all the docs, I didn't catch on. So I hopped on irc and the next thing I know, the author patiently walked me through each obvious step. He wasn't condescending, but the whole thing goes together so simply that I felt pretty stupid about not getting it on my own. There are nice people (including the author) on irc who are very patient with people who need help grasping the obvious.
This product is drop-dead simple. Part of my problem was that I was expecting it to be much more complicated.
Exactly as other post says, kdbus and the way he tried to push it raised a few warning flags. Kernel is all about trust and I just don't trust Greg after that episode.
It is all off topic since I really admire zx2c4 and this is about his product.
Edit: since Greg KH got removed from the thread title any mention of him is off topic...
For those of us who have never heard of Wireguard, I can click on the linked Wireguard image to at least see what it is. Cool, I guess? If it was a standalone link on HN ("hey, look what we made!") it would make a lot more sense, but since it's a G+ post, it's more about the blog-type content. I guess the main point is the discussion of merging it into the (Linux?) kernel tree, but... yeah.
If you haven't picked it up from context: Wireguard is a VPN that provides a virtual network interface (wg0, wg1...) which can be routed, firewalled, and so forth.
Notable bits: very little code, so it's easy to audit. Very few options, so it's easy to get it do what it does. Very simple config, as a result of the very few options.
Basically, take all the things which drive people crazy about IPsec (all the options, all the choices, the difficulty of debugging, the multiphase negotiations, the usual policy routing rather than interface routing) and make them go away by not just choosing reasonable defaults but not even providing the capability to make different choices.
Wireguard by itself has been on HN for a few times now. The newsworthy part is that one of the top kernel developers is expressing interest in bringing it into mainline, so it is appropriate to submit that post instead of the Wireguard homepage.
Happy to answer any questions you all might have about WireGuard. We're in #wireguard on Freenode, too, if you need help with things in real-ish time. I'm also mailing out free stickers to anybody who wants some: https://lists.zx2c4.com/pipermail/wireguard/2017-May/001338.... -- just email team@wireguard.com and I'll throw them in an envelope.