
WireGuard on K8s: road-warrior-style VPN server - sclevine
https://blog.levine.sh/14058/wireguard-on-k8s-road-warrior-style-vpn-server
======
tombh
I think we all understand the usefulness of a road-warrior-style VPN. But it
doesn't seem so clear what k8s is adding here?

Anyway, on the topic of scalable UDP services, does anyone have any experience
of load balancing a UDP service? Because UDP is connectionless there's no
obvious way to make UDP packets "sticky". Are there any established practices
that could help scale this k8s Wireguard service to 2 or more containers?

~~~
georgyo
Load balancing UDP isn't too difficult. However that is not the hard part
here. It is ensuring the routing happens correctly.

A client must hard code it's IP address currently, which means if it can
connect to more than one node, then it is unclear which path a response from a
server should take to get back to that client. Each VPN instance could run
NAT, but then users would never be able to talk to each other.

Wireguard makes this significantly harder than say ipsec. WG has nothing to
indicate when a client connects. And there is no dead peer detection, so you
cannot tell one a client disconnects. IE. Scripting something to update a
global routing table to say which sever has which client is near impossible.

I use wireguard daily for personal stuff. However I cannot think how I would
make it work in an active-active situation besides NAT, which I don't want.

~~~
yardstick
WireGuard proponents would probably tell you to run BGP or some routing
protocol over the VPN, maybe GRE too?

I agree with you, WireGuard makes this significantly harder than it needs to
be. Other protocols do better in this respect.

------
rektide
Worth metnioning Kilo, which is an enhancement or a CNI (container network
interface) provider that does Wireguard for Kubernetes.

[https://github.com/squat/kilo](https://github.com/squat/kilo)

~~~
jzelinskie
Yes! When you think of Wireguard and Kubernetes, you should think of Lucas! He
spends a lot of his free time experimenting with the combination of these two
technologies. At KubeCon EU Barcelona, he gave a talk about cross-cluster
networking using Wireguard:
[https://www.youtube.com/watch?v=iPz_DAOOCKA](https://www.youtube.com/watch?v=iPz_DAOOCKA)

------
vpner
GitHub has several projects that automate setting up a wireguard VPN on
various cloud VMs without K8s:
[https://github.com/topics/wireguard](https://github.com/topics/wireguard).
There's also this tutorial that sets up a VPN along with proper DNS
configuration so that DNS doesn't leak:
[https://www.ckn.io/blog/2017/11/14/wireguard-vpn-typical-
set...](https://www.ckn.io/blog/2017/11/14/wireguard-vpn-typical-setup/).

------
BillinghamJ
You can install the WireGuard tools only, without the kernel extensions etc,
with:

    
    
        apt-get install -y --no-install-recommends wireguard-tools
    

This is all you need with the server flavour of 20.04. For the minimal one,
you need a couple more.

So no need to use a builder image

------
aequitas
A few people seem to be confused why K8s is needed when you can just run this
on the OS itself. I think they miss the point that this is not a guide to
setup Wireguard using K8s but setup Wireguard if you only have/want a K8s
environment.

As the author notes: "you can run a road-warrior-style Wireguard server in K8s
without making changes to the node."

Which makes this guide ideal for me. I run a lightweight K8s flavor (K3s,
[https://k3s.io/](https://k3s.io/)) as "configuration management" on my home
server and home automation Raspberry Pi's because I don't want to mess with
OS/userland configuration or the associated tools (Puppet, Ansible, hacked
together scripts, etc) or want to maintain any OS state manually.

For my setup I just flash K3s to disk or SD card and let it join the cluster.
Everything else is configured in Kubernetes and stored nicely as configuration
files on my laptop so I have an overview of everthing and can modify/rebuild
whenever I want.

~~~
darkwater
You say you don't want to use Puppet or Ansible but you are basically using
kubernetes manifests for the same exact reason: configuration management. I
know it can be funny and I totally support it but I thought it should be
pointed out anyway.

~~~
aequitas
The problem I have with traditional configuration management is that in the
end, even if it's declerative, you are stil modifying a imperative
OS/userland. So it will collect state at some point. Things like undoing
changes with those tools is not that trivial. You have to actively reverse
them in your configuration. Which turns nice CM code into mess. Want to try
out something quick? Better not be afraid it messes up your OS/userland as
there is no simple undo.

So since I'm doing isolation in containers/Docker already it's a small step to
a lightweight Kubernetes. What Kubernetes gives me on top of that is that I
can consider everything below the application layer as a declarative API.

~~~
jatone
not really true anymore with systemd portable services. or package managers.

------
microcolonel
Well, there's road-warrior, and then there's _road-warrior_.

I've been trying out Glorytun, it does multi-path VPN with a relatively
similar wire format to WireGuard. Being mostly indoors, due to the microbial
boogaloo, I've not been trying it with the most interesting applications.

~~~
jcims
Would like to use something like this to aggregate a few DSL connections. Any
idea how well it works for that use case?

~~~
microcolonel
It seems to work well when the connections are of roughly equal speed and
stability, so that sounds like a rather ideal use case. :+ )

I think it'll need work for connections with varied performance.

~~~
jcims
Nice! Thanks for the reply, May have to give this a go.

------
wferrell
Worth taking look at [http://tailscale.com](http://tailscale.com) \- Their tag
line: Private networks made easy. No affiliation -- just like their product.

~~~
0az
My main annoyance with Tailscale is the reliance on Google. I need to refresh
my memory, but I think this makes a VLAN shared with other people impossible.

This is why I'm still using [https://zerotier.com](https://zerotier.com) \--
also no affiliation.

~~~
pot8n
Honestly that's the least of all problems and catastrophes of Tailscale. You
must have 1000% of confidence in their own servers security, if the published
public keys hosted on their servers have been tampered then the entire network
is compromised. Also, if their service is down, you will be unable to connect
to your network even if it is completely fine and working.

~~~
lima
Tailscale is open source, it should be possible to set up your own server.

The hosted Tailscale product is meant for GSuite customers who want an peer-
to-peer VPN with corporate SSO. Yes, you have to trust them - SSO login is
inherently centralized. My company uses it, it works great.

~~~
pot8n
I am not really sure you understand how it works. There is no hosted/not
hosted versions of it. You must connect your "opensource" client/agent through
their coordination servers hosted by them to host and publish the public key
to the other devices in your network and you can not skip their service. So
Tailscale is effectively as opensource as any commercial opensource VPN
client. It's entirely useless when not used with their commercial service and
users have zero control over the software unless when used with their servers.
The "open source" thing is great from a marketing and business perspective
because you basically benefit from the open source marketing and the community
thing from the unsuspecting users and enthusiasts pros without giving away
literally anything.

~~~
lima
The backend (minus the web UI?) is open source as well.

------
sandGorgon
this example uses k3s - which is the k8s distribution by the Rancher guys.
Really cool distro - simple UX. Runs equally good on a raspberry pi or the
cloud

------
stzup7
I'm not sure to understand the use case. Is the goal to replace things like
flannel? Or route all your traffic from a single gateway?

------
Legogris
Is there any reason why OP installs iproute2 and iptables not in the builder
together with the wireguard package but in the final container image?

~~~
simcop2387
The packages installed in the builders are essentially never used since it
never runs. The builder makes the files to install in the final container
during the build phase, and then gets thrown away.

~~~
Legogris
Oh, of course. Brainfart there.

------
godelmachine
While we are discussing this, I see NordVPN has also released support for
Wireguard and named it NordLynx

