Show HN: Wag, MFA and Enrollment for WireGuard

[Nullence]

https://github.com/NHAS/wag

Howdy folk,

I've been building this project as both a side project and my job for a little while now. The rationale behind it is while wireguard is a fantastic protocol cryptographically it leaves a lot to be desired when it comes to enrollment and end user device security.

Obviously instead of using an off the shelf solution like tailscale, I decided to reinvent the wheel which has honestly been quite fun with learning about eBPF, and recently clustering and HA with etcd!

The most recent version (in the docker container) contains about 6 months of very new work bringing it all from sqlite3 to etcd. So please be forgiving if it does some weird things!

Hope you all enjoy!

P.s Im not a web developer and any tips on that front to make it less teeth pullingly awful are welcome!

[ta1243]

This looks nice, but a couple of things

> curl http://public.server.address:8080/register_device?key=e83253...

> The service will return a fully templated response

It looks like the "registration" involves the server generating a private key then sending to the client, rather than the client generating a private key and sending the public key to the server.

Not only that but your example is http! Probably worth replacing that part at least in case people think http is a reasonable option.

> To authenticate the user should browse to the servers vpn address, in the example, case 192.168.1.1:8080, where they will be prompted for their 2fa code.

So when the session times out, is there any way for the client to realise this? Or does their ssh session (or whatever) just stop working?

I've on and off looked for a wireguard client which can do something like the captive portal detection on wifi. Ideally it would be an extra line in the config file (like persistentkeepalive), which does a URL pull. Could be checked periodically (like with the keepalive). If it returns "OK", then fine, if it doesn't return then there's a network problem, but if it returns a "Location" header, the client would pop up a browser at that location, allowing for session reauthentication or whatever.

I haven't found one.

[Nullence → ta1243]

Howdy! Kind of forgot that I put this up, the registration url can also take an optional pubkey parameter, so you dont have to rely on the server generating the private key for you (docs are a little lacking so I understand the confusion!)

To answer your last question, eBPF XDP which is what I use can only do PASS, DROP or REDIRECT. So I stick with the easiest possible outcome and do PASS/DROP, which means your connections will just stop working.

However you can always set up the detection yourself by adding the captive portal detection pages to your wag MFA list then the browser should do everything else for you.

Unfortunately doing interception or acting like a proxy isnt something Im looking to do with wag (which makes authorisation timeout/logout a bit easier to deal with)

Hope that answers things!

[ta1243 → Nullence]

The problem I've seen when I've looked at this is signalling to the user that the session has ended. Sure your EBF filter stops passing traffic, but the user as to know that they need to visit a page to reauthenticate

That requires integration with a client, which is a massive pain to integrate

[hnarn → ta1243]

It sounds like upon expiry, you could redirect all 80/443 traffic to the same node, serving an error page. In my mind, that's better than having to install a client.

[Nullence → hnarn]

Probably then is how you do TLS, i.e do you give your service a wildcard cert to capture any 443 traffic.

Then how do you do that for things like ssh, or other non-http based protocols.

[20after4 → ta1243]

> I've on and off looked for a wireguard client which can do something like the captive portal detection on wifi. Ideally it would be an extra line in the config file (like persistentkeepalive), which does a URL pull. Could be checked periodically (like with the keepalive). If it returns "OK", then fine, if it doesn't return then there's a network problem, but if it returns a "Location" header, the client would pop up a browser at that location, allowing for session reauthentication or whatever.

That would be really cool. I hope the author of this will consider it.

[Nullence → 20after4]

Ironically I've also been thinking about this on and off for a bit, as it is definitely one of the pain points of using Wag at the moment.

My only problem is that if you capture that route and then redirect it, or whatnot, your peers wont be able to log in to wifi in public areas like coffee-shops/libraries/etc because the route will be trying to go via a VPN which wont be returning any real data.

Such as where you should be going to log in haha

[baobun → Nullence]

Should be doable using FwMark and routing policy / nft rule?

So you'd put a fwmark on the interface , allowing you to route the VPN traffic separate from the traffic that should go over it. Then you have some mechanism to trigger when VPN is down. Curl someplace dedicated which also has a routing exception to go outside the tunnel. Detect captive portal IP. Add route (and clean up, like when online or switching).

[ta1243 → Nullence]

Detecting captive portals re-establishing is a separate, but linked issue. It's mostly solved too.

Setting aside a VPN client, your browser and/or OS will send out a http get request to a site which returns "OK" or "success". If it returns a redirection, then it will go "ahha I'm behind a captive portal" and pop up a page for you to log in.

VPNs do something similar. The MozillaVPN client for example will periodically check for reauthentication by doing a http call over the non-VPN route to a server and looking for a response saying "success". If it needs a redirect it flags up a message on the client to disable your client and reroute traffic via normal networking to allow reauthenticaiton.

What I want from a wireguard client is it to check for captive portals outside (in a situation I'm routing the entire of 0.0/0 via wireguard), but also check for reachability inside the tunnel. Have "check" and "checktime" parameters, which poll a given server (presumably via the tunnel) preset a popup to reauthenticate.

[blipvert → ta1243]

I have written a similar server (per-device client cert required which gets you mTLS to a logon page which uses OIDC the authenticate the user and enable the tunnel) but the client is the tricky bit.

I have written one in Go for the Mac which uses the command line wg from Brew and handles key gen, but it is clunky and requires sudo.

A proper native app which uses the network entitlements would be great, but it is beyond my ken

[asimops]

Have you tackled the issue of session management or are you planning to do so? Essentially wireguard key are just eternal session keys.

I would expect software that implements the wireguard transport layer to implement session management to be called a working VPN server solution. This means a second channel to the server for periodically rotating session keys, terminating sessions, changing IP addresses, configuring new routes and repeating authentication if necessary.

[gchamonlive → asimops]

I'd use firezone for that. It has an option that forces the user to login to the platform regularly. Coupling that with an external identity provider via oidc is a very solid and simple solution for session management.

[cchance → gchamonlive]

Firezone seems to have come really far from when i last used it wow... But ... I really like running headscale for most of my stuff as i prefer the p2p meshing for direct connections from server to server latency regardless of where they are.

[jamilbk → gchamonlive]

Just a quick note -- 1.0 goes a little further and rotates the WireGuard keys upon each auth session, so the private key never leaves the tunnel process memory. You need the Firezone client for that, though.

[Nullence → asimops]

Im not entirely sure what you mean by "eternal session keys" in the context of wag specifically.

While the wireguard key effectively gives you the ability to talk to the wag server, the session is effectively maintained by a map in ebpf, as to whether you've authorised or not.

So even if someone steals your private key material, they wont be able to access MFA restricted routes

[Aloha → asimops]

It seems to me that if I were building a VPN client like Global Protect but using WG, I'd have an eternal per client auth key that is used to set up an initial tunnel to the VPN controller, over that you perform auth, etc, then you're given another session key, which is valid as long as that session is. The first tunnel would disconnect as soon as you finished authenticating and got your actual session key.

[mbreese → Aloha]

Why even have the first tunnel? You could do the entire thing out of hand over a web interface or other mechanism. If the public address of the WG server (or the authn server) is known and available, then there’s no real difference in setting up the connection over a persistent WG channel vs an HTTPS service.

Both solutions are custom for that VPN, so I’d just have one WG tunnel that’s controlled through a secured (web)service. The bonus is that you wouldn’t need a hard coded initial key. (You’d still need some authentication mechanism, but it could be more than fixed public/private key pairs).

[lmz → asimops]

> This means a second channel to the server for periodically rotating session keys, terminating sessions, changing IP addresses, configuring new routes and repeating authentication if necessary.

So IPsec's IKE protocol? Why not just... use IPsec?

[d-z-m]

Do you protect against bruteforcing the TOTP code? I.e. via rate-limiting or a set amount of retries? I took a quick glance at the code and couldn't find anything to this effect.

The scenario I'm imagining is: someone opens the TOTP entry UI in their browser, opens devtools, and starts to loop through all possible TOTP codes.

[Nullence → d-z-m]

Yep! I do indeed have protections against bruteforcing TOTP codes, effectively each authentication has a number of "attempts" a user can make before their account gets locked, and an admin is then required to unlock it.

Specifically to force people to have a bit of a think as to why their device is trying to force auth to begin with, as it indicates an endpoint compromise.

[8organicbits → d-z-m]

I think thats here:

https://github.com/NHAS/wag/blob/cdbdbec3393fa86bf6c823117c8...

[Avicebron → d-z-m]

I can't speak to the specifics of this particular implementation but usually if someone has the login (username + password) to get to totp that user has already been compromised..

[asimops → Avicebron]

But MFA is there to prevent this compromise from affecting the service and alert users/admins to the compromise, right?

If you have username and password and are able to force the TOTP in the 60s window, the TOTP would be useless imho.

[yonatan8070 → Avicebron]

If a user keeps their credentials in a notebook and it got stolen, the TOTP check can be the difference between the attacker getting in, and the user being notified and changing their password

[yardstick → yonatan8070]

Unfortunately these days it’s even easier with password managers containing all three (user, pass, token)

[yonatan8070 → yardstick]

I want to believe users who use a password manager are also technically literate enough to secure it properly

[Avicebron → yonatan8070]

Me too, but my day job means I handle a bit of secops, password managers are rolled out as security tools to users operating in enterprises where things like mandating people don't keep their passwords on a sticky note on their monitor is usually step one...

[ThePowerOfFuet → yardstick]

The difference being the notebook is paper and easily read, while the password manager is... quite a bit harder.

[Nullence → Avicebron]

Oh wag doesnt use username and password auth by default. Those are only available in the OIDC integration or if you use PAM auth.

[hnarn]

This sounds very similar to Head- or Tailscale, nice to see some alternatives to managing wireguard networks. Is there a comparison anywhere to understand what functionality overlaps, what is added, what differs and what will perhaps never be implemented?

[Nullence → hnarn]

Similar in terms of it uses wireguard definitely! I havent made a direct comparison in the documentation as it's not something I'm currently going toward. This project suits my needs and is quite fun!

But I'll try and give a basic run down on the differences/similarities.

Wag is good for hub and spoke design where you want to have a hard boundary, rather than a tailscale-esque mesh where everything touches everything and then the rules define the overlay.

Both wag and tailscale add SSO integrations and effectively 2FA for securing your users.

And both of us have a way to enroll and a web UI to manage things, although I'm sure TailScale is much more polished considering I'm one guy who doesnt like web development.

As for things Im definitely not going to implement, probably interception or a TLS proxy to redirect users once their session logs out. Primarily just because doing that in eBPF is a little bit beyond me right at this second, and I dont feel like writing the DNAT/SNAT components I'd probably have to in order to get it working

[mahkoh]

>IPv4 only.

You'd think that sites choosing wireguard would have a more modern setup and might make heavy use of (self service) ULAs.

[Nullence → mahkoh]

I do plan on supporting IPv6 sometime soon, and doing something along the lines of mapping folks IPv4 addresses into private IPv6 space to reduce the risk of clashing with a users real local network.

Is there something specific you were thinking about when you mention ULAs?

[tptacek → mahkoh]

What are the ULA wins you're thinking of here?

[hiatus → eptcyka]

This kind of shallow dismissal is inappropriate, especially for a Show HN.

[lmeyerov → aragilar]

TailScale made key parts of their stack proprietary, route key bits centrally, and I'm sure other issues. And, just like with browsers, alts are good. I'd expect a lot of whitespace for a more open, secure, and usable impl.

Most people do not use TailScale. I'd encourage doing the work of understanding why, there is likely a big opportunity somewhere here.

[drio → lmeyerov]

> Most people do not use TailScale. I'd encourage doing the work of understanding why, there is likely a big opportunity somewhere here.

When you use Tailscale extensively, it becomes your new network. Now all your systems depend on a piece of software that you do not fully control. The control plane is not open source, and it is a key component of Tailscale. Headscale is a great effort, but it doesn't have feature parity with Tailscale (1). Moreover, the dedicated team at Tailscale keeps releasing amazing new functionality regularly.

That being said, if I had to buy software from a company, Tailscale would be my first pick. I respect and trust the founders and the early engineers working there.

As a side note, I'm planning to contribute to Headscale. This technology is crucial, and I want to help ensure its success.

(1) The functionality offered by Headscale is sufficient to build a robust mesh network and enjoy its benefits. Kudos to the team and to Tailscale for supporting it.

[aborsy → drio]

The security of the devices don’t depend on Tailscale coordination server, if tail lock is enabled.

An attacker with access to coordination or relay servers would be able to change whatever is in the admin console, which are basically ACLs.

Am I missing anything?

[lmeyerov → drio]

That's true today, no reason to be true tomorrow, else we would all still be on mosaic or netscape or firefox (who fired the rust/servo team afaict once they weren't as valued)

[zeotroph → aragilar]

"Numerous others" which also support MFA, have some (Web)-UI and are also open source? I only know of DefGuard[1], which aspires to do a lot more.

1: https://github.com/DefGuard/defguard discussed at https://news.ycombinator.com/item?id=36056080

[aragilar → zeotroph]

https://github.com/netbirdio/netbird seems to be completely open source (BSD), https://github.com/gravitl/netmaker?tab=License-1-ov-file#re... uses Apache for the non-pro stuff, and both of those I found by simply looking at https://github.com/topics/wireguard

This is why I asked, the phrase "I decided to reinvent the wheel which has honestly been quite fun with learning about eBPF, and recently clustering and HA with etcd" makes it sound like it's doing a bunch of cool stuff (which I want to hear about!), but the readme says nothing about those.

[Nullence → aragilar]

Ah the readme is definitely more geared to "how can I use this" rather than "how does this work.

Primarily because I want people to have a reasonably good time setting it up, rather than having to go through my explanation on things!

[Nullence → zeotroph]

Man Deguard definitely looks slick and the UI looks really nice!

I'd be super interested to know how they track "session state" as their do seem to rely very heavily on adding proxies and other additional software layers in front of the wireguard connection itself (https://defguard.gitbook.io/defguard/admin-and-features/wire...)

With wag specifically it's all just wireguard and a tiny bit of ebpf to do the management, along with tracking the external IP to determine if its time to re-challenge a user.

[helloooooooo → aragilar]

You know people can just do these things for fun right? It doesn’t have to be intended as any form of competitor.

[stavros → helloooooooo]

This is one of my pet peeves about HN. It's Entrepreneur News, not Hacker News. A hacker's first response would be "this is cool!" whereas here it's "what's the business case for this?".

[mbreese → stavros]

I don’t know if the parent was edited or not, but that’s not how I read the parent post.

I didn’t take it as malicious, but trying to understand more about this method. I’d love for the author to tell us a bit more about how it works. I’m curious about what obstacles the author hit and how they got around them.

Note: re: the flagged sibling comment. Yeah, that one doesn’t get the benefit of the doubt and was out of bounds.

[Nullence → mbreese]

Welp, I have no idea if anyone will read this as its been a little while since it was put up and is in a flagged comment, but hey I'll do a lil explanation of some of the issues I ran in to, and how it works.

How it works:

In short, Wag adds an eBPF program to a WireGuard device that it instantiates. The eBPF program uses a number of hash maps and LPM (longest prefix matching trie) maps to determine the policies that are applied traffic coming in on the wireguard device. These policies based on the ACLs defined per user/group, and contain MFA/Allow/Deny rules which require mfa, allow without auth and deny always respectively.

Wag also watches all the wireguard peers ingress IP addresses, and when an address changes it deauthenticates the user and requires the user to complete a login challenge. This is done by basically setting a bit in the maps exposed to eBPF that says "unauthorised"

Challenges:

First and foremost with WireGuard there is no good way of determining if an "external ip" i.e where the user is connecting from has changed. There was a patch set submitted for review in 2022~ that was never actually added to the kernel that would have added netlink compatibility and thus event based notification that things had changed, but alas that was never reviewed by Jason Donenfeld and has quietly died the death.

Secondly was defining multiple policies per route was quite difficult as eBPF doesnt do dynamic memory even in userland exposed maps and I wanted multiple rules per route, i.e you might allow port 80/tcp when MFA has passed but otherwise always allow 22/tcp. So to do that I had to define a maximum number of rules that could be inserted as one memory blob into the LPM map that the ebpf program would then linearly search to make its decision.

Thirdly has been making everything highly available which has been a bit of an on-going battle with ETCd mainly around how it manages cluster certificates as they dont (as of 2024 but it may be coming soon) expose the right structures to allow for dyanmic certificate creation, so you have to kind of make a wrapper around that in order to get everything going.

Im sure there are other things that I've had struggles with, but these are what come to mind immediately!

[mbreese → Nullence]

Thank you for posting this. These are the types of details that you just don’t get unless the author has really worked through these issues in depth.

Best of luck with the project!

[Nullence → mbreese]

Thanks! It's been a bit of a labor of love for quite a while, these are the big three but there are a bunch of other little things.

Like the time I had to optimise map insertion because the linux kernel does some truly insane locking when you use specific types of eBPF maps:

https://github.com/NHAS/wag/issues/84

This is slated to be improved (or has already been improved in kernel 6.8?). But for now wag sort of just side steps it in a horribly stateful way.

[aragilar → Nullence]

Ooh, this looks cool! I'm mildly curious given Windows apparently is adding support for eBPF whether you could also get this running on Windows as well.

On the patch, maybe try reposting it on the list, with the pointer to your project to see if that provokes a new review?

[Nullence → aragilar]

Haha, me and a friend were both looking at the windows eBPF project and wondering if it might work. Im not really looking to support windows, but if someone comes to me and tells me it works I'll pop it on the read me

[stavros → mbreese]

Oh, I'm not saying it's malicious, it's just a mindset of "creating a thing must be externally motivated" vs "we make things because we like making things".

[mbreese → stavros]

I agree that that dichotomy is on display here quite often. And it isn’t a good mindset. Making for the sake of making and learning should be celebrated.

I just didn’t get that vibe from the now flagged comment by @aragilar. I saw it as a genuine curiosity about the design choices. Maybe I was wrong.

    “¯\_(ツ)_/¯“

[linsomniac → qweqwe14]

You sound like a fun guy to have at parties. :-)

[mynameisvlad → qweqwe14]

That was quite clearly expressed in the OP:

> Obviously instead of using an off the shelf solution like tailscale, I decided to reinvent the wheel which has honestly been quite fun with learning about eBPF, and recently clustering and HA with etcd!

Just because you don’t see or don’t care to see the point doesn’t mean that it doesn’t exist. And just because it’s not fun to you does not mean it’s not fun to anyone.

As everyone else has said, people can choose to have fun with their free time and what they do with it is up to them. “This isn’t a viable competitor” is an extremely shallow dismissal of one’s work when the motivating factor isn’t even remotely making a competitor. Which some people couldn’t give two shits about.

[pvg → qweqwe14]

It's not fun

If it's not fun or interesting for you, just read something other than Show HN's. This sort of meta is totally offtopic in an actual Show HN which is about the work being shown rather than your personal feelings about the category.

[gremlinunderway → qweqwe14]

I play a video game that I'm sure someone has speedrun way faster than me, and got way more points and did it with way more efficiently than me.

I still play video games.

Making a Tailscale replacement doesnt sound fun for me, but for some I guess it is? Why shit on that? Who cares?

[tristor → qweqwe14]

I am a cynical and curmudgeonly person so I am usually want to agree with your take, however I think you have missed something critically important:

Most technical people learn by doing. There is value in doing something even if you throw it away, even with hardware in meatspace, because it gains you knowledge and intuition about how the better/established thing works. This knowledge can be golden.

I have a very deep understanding of a lot of systems that most people never touch because they “just work” and are well implemented. I have that understanding exactly because of garbage toy projects (most of which I never shared). This knowledge has made me invaluable in many projects/jobs in my life, because nobody else bothers to understand those systems.

In fact, one of the most obvious (although perhaps not most efficient) ways to do clean room reverse engineering is to treat a system as a modeled black box and try to naively reimplement it using your own assumptions and then benchmark your outputs against the black box model. This reveals significant knowledge about the workings of the black box.