
The trouble with VPN and privacy review sites - dngray
https://blog.privacytools.io/the-trouble-with-vpn-and-privacy-reviews/
======
dguido
I appreciated that Wirecutter's review of VPN services began with
investigating which ones had contracted for 3rd party security reviews. This
presciently excluded many providers that were all about marketing, like
NordVPN. I think this is the best approach rather than the litany of mostly
useless criteria that's cataloged on ThatOnePrivacySite. You want to extract
out the high signal, and "will my VPN service get hacked and all my traffic
get leaked?" should probably be your first question when choosing a service.

[https://thewirecutter.com/reviews/best-vpn-service/#how-
we-p...](https://thewirecutter.com/reviews/best-vpn-service/#how-we-picked)

Full disclosure, I'm the author of AlgoVPN, a set of scripts for hosting your
own VPN rather than using a 3rd party service, and was interviewed by
Wirecutter for their article. You should use Algo if you're at all capable of
doing so:
[https://github.com/trailofbits/algo](https://github.com/trailofbits/algo)

~~~
madamelic
In terms of privacy, isn't running your own VPN pointless?

I mean, it is basically just changing your IP and putting an additional hop
in. You aren't mixing with other's traffic, making it very easy to fingerprint
you.

I guess this is all dependent on someone's threat model, but I am not really
sure if there is any benefit of running your own VPN besides being slightly
more sure your VPN provider or someone who hacked your VPN provider isn't
watching you.

~~~
rbritton
It provides you protection on a hostile local network, such as a hotel or a
restaurant. Like you said, it does not achieve the anonymity level a public
VPN would.

~~~
yjftsjthsd-h
> protection on a hostile local network, such as a hotel or a restaurant.

Or an American ISP.

~~~
nickpsecurity
Fourteen Eyes:

[https://restoreprivacy.com/5-eyes-9-eyes-14-eyes/](https://restoreprivacy.com/5-eyes-9-eyes-14-eyes/)

------
tptacek
This appears to be a story about how commercial VPN review sites are
untrustworthy because they accept advertising from commercial VPNs, written on
and with promotion for a site that provides reviews for commercial VPNs and
accepts sponsorship money from those VPN providers.

A pox on the entire commercial VPN "industry". They all deserve each other.

While I don't think you should use _any_ of these commercial VPN servers, I'll
give props to WireCutter for at least attempting to do a serious job of
impartial reviews.

~~~
dawidpotocki
PrivacyTools doesn't have a deal with any VPN company or any other company.
They can become our sponsors, but they won't get listed as our recommendation
on the website [1] (we don't even have any sponsor at the moment), also we
can't profit from it personally and everything is transparently shown on
OpenCollective [2]. We even give a huge red warning that VPN is not going to
help a lot and that you should probably use Tor instead [3]. Also we removed a
lot of VPN services from our site, some time ago [4][5].

[1]
[https://www.privacytools.io/sponsors/](https://www.privacytools.io/sponsors/)

[2]
[https://opencollective.com/privacytoolsio](https://opencollective.com/privacytoolsio)

[3]
[https://www.privacytools.io/providers/vpn/](https://www.privacytools.io/providers/vpn/)

[4]
[https://github.com/privacytoolsIO/privacytools.io/issues/113...](https://github.com/privacytoolsIO/privacytools.io/issues/1139)

[5]
[https://github.com/privacytoolsIO/privacytools.io/pull/1174](https://github.com/privacytoolsIO/privacytools.io/pull/1174)

~~~
tptacek
Why can't every VPN review site say the same thing? Serious review sites don't
accept sponsorships. This one does. Should it be taken seriously? I don't take
it seriously.

~~~
dngray
> _Why can 't every VPN review site say the same thing? Serious review sites
> don't accept sponsorships. This one does. Should it be taken seriously? I
> don't take it seriously._

1\. We currently get _no_ money from _any_ VPN provider (our site makes other
recommendations too that are not related to VPN services), our finances are
very transparent:

• [https://blog.privacytools.io/privacytools-io-joins-the-
open-...](https://blog.privacytools.io/privacytools-io-joins-the-open-
collective-foundation/)

• [https://opencollective.com/privacytoolsio#section-
goals](https://opencollective.com/privacytoolsio#section-goals)

2\. Being a part of the sponsorship program does not get you on the website,
you must still meet the criteria which a VPN provider could do for free (so
there's no incentive for them to pay us anything).

3\. We don't use referral links

4\. No single member of the team can add/remove things (everything is also
logged in git commit logs). Pull requests also require more than 2 members to
sign off. Technically jonaharagon as owner could add things, but it would be
pretty suspicious if new VPN providers started appearing without any
discussion.... lol. I know I'd be asking questions.

~~~
tptacek
Or, like reputable review sites, you could just not accept sponsorships from
vendors.

------
anon9001
I use a VPN for privacy, which is great, but routing my traffic through it
will exclude me from sites that try to block VPNs (mostly streaming services).

What I really need is Cloudflare's WARP via wireguard config. I love the idea
that they'll shield me from my ISP but still provide my real IP to service
providers.

I _can_ do this right now with a hack someone wrote
[https://github.com/maple3142/cf-warp](https://github.com/maple3142/cf-warp)
but I don't want to anger Cloudflare.

Cloudflare, if you're listening, is it ok to extract wireguard credentials
from your app and use them on my whole network? I'll gladly pay the $5/mo, but
I don't want to be banned from Cloudflare or do something you may construe is
illegal by extracting keys from your Android app.

~~~
mkbkn
You could email them and ask for this feature.

------
pytester
The problem with free review sites is that once you've built up trust as an
honest, objective reviewer, the most effective way to profit on that trust is
to violate it.

~~~
dcolkitt
This may be true. But that doesn't mean the best strategy is to completely
burn your reputation. Even if the only way to profit is to sell-out, it's
usually better to sell-out slowly and collect much more sustainable revenue
over a longer time frame.

There are a lot of equilibria where most free review sites are mostly, but not
completely, trustworthy. With enough review sites in that model, an end-user
can effectively triangulate the objective truth with arbitrarily high
certainty.

------
fmajid
This article misses the most vital point: VPN providers are asking you to
trust them, and there is no way to verify that. That's why I think DIY (e.g.
Algo or Streisand) is the only way to go.

~~~
kylec
This is why I desperately want Apple to build a VPN service. They are already
committed to privacy, and they've got a lot more to lose than some fly-by-
night VPN service.

~~~
Mindwipe
> They are already committed to privacy

They're also already committed to censorship, so I don't fathom how they'd run
a VPN service.

------
jmarbach
I started wondering which VPN might be best to use, then I realized, there's
little comparison to _your own_ server. Outline VPN is a neat open source tool
that makes this possible with no server setup or maintenance required:
[https://getoutline.org/en/home](https://getoutline.org/en/home)

~~~
justaj
The point of using a commercial VPN is that you can share an endpoint with
multiple users. Thus "blending in" with the crowd. If you choose to use a VPN
endpoint that only you use, you lose all privacy benefits except the one
against your ISP.

------
tcd
The best approach I feel is how [1] does it.

A list, what features each provider has, and leave it to yourself to make the
judgement. If you're being told why it's good there is bias involved somewhere
along the line.

Of course, you need to understand whether the site has updated their
information and presenting it truthfully, which should be easily verifiable.

[1]: [https://thatoneprivacysite.net/](https://thatoneprivacysite.net/)

~~~
BanazirGalbasi
That's mentioned in the article, but it does bring up the caveat that many
consumers are just looking for an answer, not the information they need to
form their own answer. That's why VPN review sites are so common, it's a quick
and easy response rather than a list of features to sort through and compare.

------
anon9001
If you're looking for VPN review sites, this is the only one with credibility
IMO: [https://thatoneprivacysite.net/#detailed-vpn-
comparison](https://thatoneprivacysite.net/#detailed-vpn-comparison)

My personal recommendation is AirVPN, but I wish they supported wireguard.

~~~
pnutjam
AirVPN is the best, but PIA is ok if your not doing anything illegal. They
will shield you from any copyright liability. It all depends on your threat
vector.

I think alot of this vpn hand-wringing is really just meant to discourage vpn
usage in general. There have only been afew cases of paid vpn services giving
up user information and they are well publicized.

~~~
edjroot
Funny you mention PIA as they've just been acquired by a reportedly shady
group.

[https://www.techradar.com/news/cyberghost-owner-buys-pia-
for...](https://www.techradar.com/news/cyberghost-owner-buys-pia-for-
dollar955m-to-create-vpn-giant)

------
jayalpha
Just a SEO blog post that posts something obvious.

They mention ThatOnePrivacySite.net but criticize him:

"Here's the difference. They include virtually every provider — the good and
the bad — and present them at equal value to sort through. Instead of
providing their readers with answers, they provide them with information that
can be used to deduce their own recommendations, based on their values as an
individual. "

1st: providing them all guarantees that there is no conflict of interest

2nd: "Instead of providing their readers with answers" You can not provide
this answer since there a tons of reasons to use a VPN

"Your VPN provider should not be hiding away in Panama controlled by anonymous
leadership."

This is also bullshit. In fact, some of the most resilient VPN provides
provide no legislation at all. They only exists in Cyberspace. "Sue us!"

I actually have written the ThatOnePrivacySite.net guy and asked him to put
this VPN on the list: [https://www.rapidvpn.com/setup-vpn-softether-
ubuntu](https://www.rapidvpn.com/setup-vpn-softether-ubuntu)

It is the only VPN that I am aware of that works out of the box with
softether. I have not tried it yet. I currently use Astrill. Astrill is not
cheap but works pretty well to circumvent censorship. A disadvantage of
Astrill is that it often leaks DNS like a motherf....

This should prevent DNS leaks on Linux if UFW is installed.

ufw default deny outgoing

ufw allow out on tun0

ufw allow out on tun0 to 84.200.69.80 port 53

~~~
dngray
> _It is the only VPN that I am aware of that works out of the box with
> softether. I have not tried it yet. I currently use Astrill. Astrill is not
> cheap but works pretty well to circumvent censorship. A disadvantage of
> Astrill is that it often leaks DNS like a motherf...._

Should also keep in mind a few years ago Astril was using weak keys like
ExpressVPN. That really makes me wonder what they know about running VPN
servers. I think you only get one chance with your reputation on things like
this.

[http://blog.zorinaq.com/my-experience-with-the-great-
firewal...](http://blog.zorinaq.com/my-experience-with-the-great-firewall-of-
china/)

------
zipwitch
Everyone is commenting on privacy, but isn't the main use for VPNs (at least
in the US) to avoid the consequences of digital piracy?

~~~
stubish
Piracy is a case where you can't afford to give up your privacy for
convenience, and common in the US, so probably yes. It is different in other
countries. Lots of countries don't care about digital piracy enough to enforce
restrictions. Pornography, access to uncensored social media, or just being
able to email people outside of your firewall become the major use cases. It
is still fairly rare to find people who are prepared to pay the costs for
privacy just because they want privacy. I think that would change if the
friction was reduced, such as Apple, Google or Microsoft including free VPN
services built into their products.

~~~
presumably
Google already includes free VPN services for Android, although limited to
public/open WiFi [0] or Fi users [1].

0: [https://www.howtogeek.com/275474/how-to-use-androids-wi-
fi-a...](https://www.howtogeek.com/275474/how-to-use-androids-wi-fi-assistant-
to-keep-your-phone-safe-on-public-networks/)

1: [https://techcrunch.com/2018/11/13/googles-project-fi-gets-
an...](https://techcrunch.com/2018/11/13/googles-project-fi-gets-an-improved-
vpn/)

------
MrGilbert
I had to giggle a bit. The article claims that "[...] you'd have to scroll
down to #6 before you found a provider that wouldn't pay them [...]" on an
"unnamed review site". The service in their list at #6 would be "Mullvad".

Looking at their own list[1], "Mullvad" is the only VPN provider listed at the
top of the list under "Recommended VPN Services".

Just something that caught my eye and which I considered an interesting
coincidence.

[1]:
[https://www.privacytools.io/providers/vpn/](https://www.privacytools.io/providers/vpn/)

~~~
dngray
Yes and this is because it meets the criteria
[https://www.privacytools.io/providers/vpn/#info](https://www.privacytools.io/providers/vpn/#info)

specifically the item that got it there was that they had external auditing.

> Mullvad's VPN clients have been audited by Cure53 and Assured AB in a
> pentest report published at cure53.de. The security researchers concluded:
> [https://cure53.de/pentest-report_mullvad_v2.pdf](https://cure53.de/pentest-
> report_mullvad_v2.pdf)

We would like to see more VPN providers do this. Then we could have more good
choices to choose from. A lot of the larger ones could certainly afford it.

~~~
gitanovic
The audit looks actually legit, but still they are located in a 14 eyes
jurisdiction, not really the best for privacy

------
billpg
In a world where HTTPS was the exception, I could see a use for this sort of
service. Now not-HTTPS is the exception, not so much.

Privacy from your ISP? Okay, but I've replaced that problem with privacy from
my VPN provider. Is that a better problem? Is my VPN provider going to exploit
me less than my ISP would have?

Geographic restrictions? That's a genuine benefit. Alas, it would end up as a
bit of an arms race as websites that really don't want me to visit would start
blocking VPN providers.

~~~
onychomys
> Is my VPN provider going to exploit me less than my ISP would have?

Well, probably the answer is yes, since we have many years of experience with
the terrible behavior of ISPs. VPNs have a much smaller userbase, so I suppose
they have less of an opportunity to screw you over, but come on, even the
worst of them has to be better than something like Comcast.

------
whycombagator
In addition to PrivacyTools.io and ThatOnePrivacySite/Guy, RestorePrivacy.com
is a site I've used for privacy related research.

~~~
commoner
Restore Privacy uses affiliate links. This article criticized recommendations
from sites like Restore Privacy, because the financial compensation creates a
conflict of interest.

~~~
whycombagator
Sure. I read the article and understand the concern. But to me, RestorePrivacy
is quite different than the spammy ones the article alludes to.

I think Sven does a decent job of analyzing each service/offering and
presenting the information in an approachable way.

That being said, it is wise to take his rankings/thoughts on each service with
a grain of salt.

I just wanted others interested in this topic to be aware of another resource
that I have found useful.

~~~
commoner
Restore Privacy's NordVPN review doesn't mention its 2018 security breach,
which was widely covered in the news:

[https://restoreprivacy.com/nordvpn](https://restoreprivacy.com/nordvpn)

[https://www.cnet.com/news/after-the-breach-nord-is-asking-
us...](https://www.cnet.com/news/after-the-breach-nord-is-asking-users-to-
trust-it-again)

Their "Best VPN List" doesn't mention it, either. That's extremely damning to
Restore Privacy's credibility as a review site, and highlights how financial
conflicts of interest can degrade the quality of a site's content.

~~~
whycombagator
Maybe it’s been added in the last 8 hours, or perhaps you never actually
checked.

But it’s mentioned, right at the start, on the page you link:

> In October 2019, news broke about a NordVPN security incident.

Which links to a full article on it[0].

He sort of downplays the hack, which then led me to read the article you
posted. And the TechCrunch article it mentions.

They take a more “trust is compromised” stance. So to reiterate:

> it is wise to take his rankings/thoughts on each service with a grain of
> salt.

[0] [https://restoreprivacy.com/nordvpn-
hack/](https://restoreprivacy.com/nordvpn-hack/)

~~~
commoner
The new "Trust issues?" paragraph was added to the review after I posted my
previous comment. I checked before I posted, and it was not there.

[https://web.archive.org/web/20191118050427/https://restorepr...](https://web.archive.org/web/20191118050427/https://restoreprivacy.com/nordvpn/)

[https://restoreprivacy.com/nordvpn/](https://restoreprivacy.com/nordvpn/)

The most recent Wayback Machine archive (November 18) shows that the "Trust
issues?" paragraph wasn't in the NordVPN review until very recently. Thanks
for getting the paragraph added in, because transparency is important.

However, you might want to consider using the pronoun "I" or "we" instead of
"he", because astroturfing is not a transparent thing to do. It doesn't take a
genius to see that you're affiliated with Restore Privacy just as Restore
Privacy is affiliated with NordVPN.

~~~
whycombagator
Thank you for pointing that out. My mistake, sorry about that - I should have
checked there before replying to your original comment.

Interestingly, it looks like he doesn't censor/hadn't censored the couple of
comments mentioning the breach on that page.

His article on the breach/hack was published in Oct. But his Nord VPN review
was published/updated the month prior (from your waybackmachine link). So a
month later.

It's plausible to me that he just never got around to updating the original
post/review. But apparently your comment prompted him to do so.

However, he hasn't bothered to jump in this thread and comment that was the
case. So although I think RestorePrivacy is still a useful site, perhaps a
larger grain of salt is needed.

> However, you might want to consider using the pronoun "I" or "we" instead of
> "he", because astroturfing is not a transparent thing to do

Not Sven and not astroturfing. Merely suggesting what I thought was a decent
privacy resource - in a related topic's thread.

------
mirimir
Some years ago, I looked at a bunch of these VPN review sites over time,
relying on archive.org captures. At any given time, there were just a few
distinct sets of rankings. Some of that could have reflected shared ownership.
But it likely also reflected changes over time in advertising budgets.

------
petercooper
It's the new "Web hosting review" site. 10-15 years ago there were hundreds of
similar commission-funded sites recommending shared hosting providers, all
with the same problems. Where there's a commission, there's someone ready to
write a good review :-)

------
stanislavb
I'd recommend giving a try to "sshuttle". It's easier to use than it seems,
and it's been working flawlessly for me.

------
ryanmercer
TLDR: they're just like _every other review site_ and are either run by the
companies rated best or are affiliate link sites.

(I accurately guessed the above before even going to the article...)

~~~
basch
Mattresses seems like another one so over polluted by affiliate links, you can
barely find a real comparison. (sleeplikethedead.com seems pretty good.)

On its face, affiliate reviews are ok if the company has integrity, like
Wirecutter attempts. If they pick a profitless product as first over one that
makes them money.

Credit card reviews are another one that have gone off the chain. There are
thousands of identical sites ranking the same cards.

~~~
fmajid
Undisclosed affiliate relationships are against the law in the US:
[https://www.ftc.gov/tips-advice/business-
center/guidance/ftc...](https://www.ftc.gov/tips-advice/business-
center/guidance/ftcs-endorsement-guides-what-people-are-asking)

~~~
jonaharagon
The problem with a lot of these affiliate sites (which I alluded to in the
conclusion of the article but perhaps didn't spend quite enough time on) is
that they provide a small disclosure of their relationships in their footer or
in the article. But they do it as inconspicuously as possible to avoid the
drawbacks of disclosing anything.

I have a lot more respect for the sites that prominently disclose their
relationships, like Wirecutter. Most of these sites are a business, they've
gotta make money somehow. But IMO most readers aren't seeking out such
disclosures automatically when they see a "review", so the hidden-in-the-
footer nonsense is entirely useless.

