This seems pretty bad. I was pretty leery of Tailscale having too much control over my network when I tested it out and now seeing this security notification reinforces my decision to use vanilla WireGuard and Nebula for my home and datacenter use cases.
Every time it gets mentioned here I wonder the same thing. Eg in the discussion of their last vulnerability they said they knew the vulnerability had not been exploited maliciously because they did not detect it on their end when their software phones home to them.
> Why on Earth would I want to use a security product that phones home... Regular WireGuard works perfectly fine for me.
Then you should continue using regular WireGuard. However, modern industry is plagued by an endless war with vulnerabilities, exploits, and malicious insiders. Even with adequate staffing, it's like a dam you're constantly patching to prevent leakage. We have to log everything, often offsite, and often into immutable storage. When the dam does eventually leak, we have to know how much and how it started. The logging is a feature to me.
Tailscale is building a service that doesn't require me to run and maintain a centrally connectable server, one that ties into a single-sign-on solution, one that logs activity, one that's introduced a system in which I don't even have to trust their control plane exclusively (Tailnet Lock). Just the seemless integration with Azure AD has saved maintenance time over NPS+Radius+ADConnect+OpenVPN.
Wireguard is great, I'm using it for all my site-to-site still (and it blows OpenVPN out of the water). But Tailscale has replaced all my client vpns for good reason.
Really? I use the (official?) Android and it worked for me fine first time, and has for years.
I think maybe you ran into a UI bug where it wouldn't work unless a subnet definition ended in a 0 (ie defining 192.168.0.1/32 would cause it to silently not boot up).
I also would like to be able to use my home lan DNS server for home lan hosts.
It might not be a Nebula problem, but to do this I need to use DNS-over-tls in Android, through the Nebula tunnel.
Edit: noticed you were referencing the WIREGUARD phone app. I'll go try harder.
> When the dam does eventually leak, we have to know how much and how it started. The logging is a feature to me.
Logging is a feature. Logging to some random 3rd party is not. Sure, if the 3rd party provides the service itself they need the logs to make it better but if stuff stays within your own infrastructure it should not phone home. And VPN service for enterprise certainly shouldn't have controller hosted on outside of company's own infrastructure
Well it's not a "random" 3rd party, it's a company you decide to rely on for your security. You pay, they offer you a service. If you don't trust them, that's fine, the product+service isn't for you. But it's not a "random" 3rd party.
>Why on Earth would I want to use a security product that phones home..
I share your opinion, I don't want my security products to phone home, but the question was answered earlier : 'you' want a product that phones home so that the central data collection group can make statements like "vulnerability was not triggered or exploited." -- otherwise the onus is on the data holder to make similar assessments.
in other words , the proprietors of wireguard cannot make statements regarding their entire userbase. (This may or may not be a good thing.)
WireGuard is a pain in the ass to setup though. Also, their Windows client is no longer supported and has showstopper bugs (tunnels disappear until UI is launched and they are manually reactivated there).
I mean, if they had proper bug tracker instead of a mailing list, we could see the answer to that right away. But I am referring to https://lore.kernel.org/wireguard/CAHmME9pbY0Cgbj5JbTVvsxpkd... which I believe just made me travel to a remote location to restore connectivity.
> Why on Earth would I want to use a security product that phones home...
Same as outsourcing security: When you want a team of professional security engineers take care of it.
(In that case, you may want to wish they are ethical, not overworked and actually care about your servers…)
Same as outsourcing security: When shit hits the fan, you have somebody else to point your finger at. No inconvenient questions to ask withing your org, potentially having to blame someone you like on a personal level etc.
Worst case, you have to ask the guy who green-lit the outsourcing some inconvenient questions, but if the company you outsourced to is big enough, you can easily take the "nobody ever got fired for buying IBM" escape route.
>>Why on Earth would I want to use a security product that phones home..
that ship sailed a long time ago in enterprise networking. Almost all major vendors now have Cloud Control, and all of them phone home for various things
>How do you connect back to your home from the outside?
>- your ISP IP change all the time
I have a static v6 IP on the WG server (home router, running OPNsense).
Even if I did have a changing IP, I have a programmable DNS server that points to the WG server's IP.
>- You need to open port
One time setup on the server.
>- You need to setup wireguard
Installed just like any other distro package, plus one-time setup to generate the key and import it and the server's public key into systemd-networkd / NetworkManager.
>- You need to manage auth / authz
You generate a key on each device and register the public key with the server. There's literally nothing else to it.
>Lot of things can go wrong and it's a lot of things to setup / maintain properly.
I set it up about two years ago and it's been working unchanged since.
>I have a static v6 IP on the WG server (home router, running OPNsense).
Excuse my snark, but not everyone has a static v6 IP. I don't.
>One time setup on the server
Unless you happen to be behind a CGNAT or you're on a mobile network or or or or...
>Installed just like any other distro package, plus one-time setup to generate the key and import it and the server's public key into systemd-networkd / NetworkManager.
And if that won't work you're gonna be stuck debugging the network setup. I certainly do always end up debugging the VPN network stack eventually.
>You generate a key on each device and register the public key with the server. There's literally nothing else to it.
This won't scale to more than like 5 devices without being a major work item if a key was compromised or needs to be rotated (what if it turns out the RNG device was bad on your kernel at the time? Happened to SSH Keys on RPi's).
>I set it up about two years ago and it's been working unchanged since.
>Excuse my snark, but not everyone has a static v6 IP. I don't.
My ISP doesn't have it either. (They've been promising it for 2 years but keep delaying working on it.) I have an HE tunnel.
Also you ignored the sentence after that, which I had written to hopefully preempt this response.
>Unless you happen to be behind a CGNAT or you're on a mobile network or or or or...
Your server needs to have a globally reachable IP, yes.
>This won't scale to more than like 5 devices without being a major work item if a key was compromised or needs to be rotated (what if it turns out the RNG device was bad on your kernel at the time? Happened to SSH Keys on RPi's).
Depends on which key, of course. If a device's key needs to be rotated, you only touch that device and the server. If you rotate the server's key, then yes you touch every device.
>And if that won't work you're gonna be stuck debugging the network setup.
>if a key was compromised or needs to be rotated
>Not everyone is that lucky.
Sure. I know these things that happen never or rarely (currently, never), so I'm okay with doing them manually when they happen instead of outsourcing. I don't make decisions for other people or claim that what I do is best for other people. You should make decisions for yourself by evaluating the pros and cons for yourself.
> Even if I did have a changing IP, I have a programmable DNS server that points to the WG server's IP.
I believe that WG resolves the domain only once, upon config load. Not much help if the server IP changes after starting the WG client. Up to the user to realise and stop/start- not so convenient for eg embedded WG clients in travel routers
If the client is still at the same address it was last time it contacted the server and the server sends data to the client it will be transparently handled. This means that persistent keepalives should be used on both sides. It also still leaves open the possibility of the connection being lost if IP addresses on both sides change during the period between the keepalives or if there is any kind of network disruption preventing traffic from the server to the client during the period the server IP address changes. I had this become an issue for me a few times when I ran a wireguard server at home.
This won't work if the server is being blocked. My mobile ISP blocks incoming UDP traffic unless the phone establishes the "connection" first. Since change of the server's public IP means that "connection" is broken, all unrelated UDP packets never arrive.
The NetworkManager WG integration re-resolves. systemd-networkd doesn't, though it has an open PR for it, though that PR has been stalled for a long time.
A VPN solution for your home network is a very different thing than what is needed for any 50+ employee org. Bare wireguard is pretty useless in a business environment; this is why Tailscale is growing so fast.
Why would VPS provider need to be trustworthy at all? They just need to be reliable. Having VPN doesn't absolve you from securing hosts within it, like you would in any other network. VPN just simplifies the networking.
And if they're not reliable, VPS providers are much easier to switch than "Tailscale".
I was not claiming Tailscale has to be trustworthy.
Though, thinking about it, it's a privileged software running on every node in the virtual network, that is controllable from outside (cloud). So yeah, it needs to be way more trustworthy than some static wireguard setup you setup on a VPS.
What scenario are you imagining where your vpn is compromised through Hetzner or Digital Ocean for example? This is a standalone server set up just for you, not sure how tailscale works but does it spin up a segregated server for each customer without a single point of failure?
So they can do analysis on the logs and proactively fix vulnerabilities before they are exploited? I'm sitting here in shock wondering if you don't realize what you typed, or your implicitly saying you don't want a software company to have useful analytics to fix their product. The second option is understandable, but at least acknowledge the utility of logging.
Also, I'm pretty sure you'll find detailed discussion of why people use Tailscale when there are options like vanilla Wireguard and OpenVPN if you google it or check their homepage. It fits many peoples' usecases.
>I'm sitting here in shock wondering if you don't realize what you typed
After you're done being shocked, maybe you should read my comment more carefully. I didn't say "Tailscale shouldn't have logging". I didn't even ask "Why does Tailscale have logging?" I asked "Why would I want to use a security product that phones home?"
I don't make decisions for Tailscale. If they want to have logging, analytics or free puppies, they're welcome to. I make decisions for myself, and I've decided that I wouldn't use any security product that phones home.
I'm so glad you 1. read the article 2. precisely understood the vulnerability 3. carefully analyzed the situation 4. came up with a high quality analytical comment
Ding, ding, ding! A lot of security/compliance folks like to pay money for scapegoats. It certainly helps with job security. Worst case scenario? Find another vendor.
> Or because SaaS is easier to use and requires little or no maintenance.
YUP
We recently deployed Tailscale in our organisation; I had looked at Wireguard very closely but decided that the ease of use of Tailscale meant we could get up and running much more quickly and easily and at relatively low cost (small team).
We'll re-evaluate as we move forward & probably need to deploy it to more users, but it was the difference between a few minutes setup to get all users into it versus several hours of figuring out the ins and outs of Wireguard & then getting the team onboarded.
While any security incident is frustrating, they are inevitable, and the only way to really judge an organisation is by its response. Tailscale's response here gives me more confidence, not less.
Just the default Google Workspace one for the moment. Only a small handful of users at the moment & didn't need anything fancy.
We are probably going to be looking at a redo of our whole auth at some point in the next 6-12 months as we plan to grow in size & move a lot more people into Tailscale.
Nebula doesn't really have a control server of this sort, it largely uses a CA to do the node authentication and a coordination server that helps nodes get introduced and NAT bust, more like the DERP server for tailscale.
The Nebula equivalent of this would be the Defined Networking folks, who do run a control server more akin to Tailscale. They say they are moving slow to focus on security, and I haven't heard of vulnerabilities like Tailscale, but also I think Defined Networks is much, much smaller in terms of users, so it may be a time will tell situation.
IMO it's more about agency. With SaaS people think "they had a bug and there's nothing I could have done to prevent it or expedite the fix" but with on-prem software they think "once I discover a bug I can whip my people to have it fixed within an hour". This is not true of course.
