
DoH: Anti-Competitive and Network Neutrality Aspects - stock_toaster
https://blog.powerdns.com/2019/12/03/doh-anti-competitive-and-network-neutrality-aspects/
======
veonik
The EU might be doing a better job here-- but is this suggesting that I trust
my US ISP's DNS servers? The ISP that injects HTML into my HTTP requests? The
same ISP known to hijack DNS requests?

Don't get me wrong, the issues raised by OP are valid and need to be
addressed-- already we see the strangely privacy-antithetical behavior of
using DoH to get around CNAME tracking in adblockers (using Cloudflare no
less) --but I don't think legislation and regulation have been effective so
far. Expecting that to change in the near future seems like a pipe dream.

~~~
zrm
> The EU might be doing a better job here-- but is this suggesting that I
> trust my US ISP's DNS servers?

That's the false dichotomy. You shouldn't use either one.

The issue is that if you've already made your choice, e.g. by having your DHCP
server give out the DNS you actually want to use or configuring it in your
operating system, then having Mozilla default to ignoring your choice and
overwriting it with Cloudflare is wrong.

And it would be one thing if it was something like people selling internet
gateway devices that do this by default -- that's a great idea because it gets
you away from the ISP's DNS by default but still provides a single place to
set it the way you want it, and still allows the owner of an individual device
to override it device-wide if desired.

Putting it in individual applications is the opposite. It means you have a
full time job just changing it back to the way you want it in every
application on every device, and even then you're liable to miss some.

~~~
sciurus
> It means you have a full time job just changing it back to the way you want
> it in every application on every device, and even then you're liable to miss
> some.

Hopefully not. Firefox has some techniques for disabling DoH that rely on
computer or network-wide changes. If other applications also adopt these or
similar techniques, you'd only have to make the change in one place to turn it
off for every application running on any computer on a network.

Specifically, as I understand it, Firefox disables DoH if

1) Windows Group Policy is used to manage a computer

2) A canary domain is blocked

[https://support.mozilla.org/en-US/kb/dns-over-https-doh-
faqs...](https://support.mozilla.org/en-US/kb/dns-over-https-doh-faqs#w_how-
will-doh-impact-enterprises-with-custom-dns-solutions)

(Disclosure: I work for Mozilla, but not on the Firefox browser)

~~~
zrm
> 1) Windows Group Policy is used to manage a computer

This is solving the problem only where it least matters. If you're using Group
Policy then you can configure DNS using Group Policy anyway, so any default
there is already easy to override. Where the default really matters is on the
devices where the only central management is DHCP, because those are the
devices that require manual interaction to fix an application that defaults to
ignoring the DHCP configuration.

> 2) A canary domain is blocked

That could work, but what happens when the ISPs start blocking the canary
domain? The FAQ says they'll monitor it but that doesn't tell you how they'll
address it when it happens.

~~~
yellowapple
> but what happens when the ISPs start blocking the canary domain?

Put your own DNS server in front of it that resolves the canary domain, I
guess.

The absurdity of rolling a DNS server solely to unblock a domain blocked by an
upstream DNS server specifically so that an application can resolve that
domain and consequently not actually use your DNS server is not lost on me.

~~~
zrm
Technically correct -- the best kind of correct.

Though obviously if you've got your own DNS server in front of the ISP's then
you could just configure _that_ to resolve names however you like. Which would
then typically cause it to accurately resolve the canary domain by default and
thereby cause the application to not use it. Hmm.

------
souterrain
> Note that some countries have an underdeveloped ISP market, with large
> fractions of the population having no choice of broadband service provider.
> Regulation is then of the utmost importance to keep everyone honest, but in
> some of these countries the regulator has been captured by industry and is
> no longer very effective. This mostly goes for the US.

I think this is the key motivator for a move to DoH. Large US ISPs can’t be
trusted.

~~~
baggachipz
> Large US ISPs can’t be trusted.

It's certainly unfortunate that we've gotten to this point. The net result is
that nobody can be trusted: Not ISPs, not Cloudflare. There is a clear
conflict of interest in Cloudflare's strategy, and their "aww shucks, we're
just being the good guys" attitude is dishonest. At this point, one or many
not-for-profit DoH providers (with easy selection of service in your browser
or OS) would be the only true solution to this issue.

~~~
tptacek
So don't trust Cloud Flare. DoH in no way depends on them. You can simply run
your own DoH resolver.

------
fhrow4484
I don't seem to understand the whole thing. To me the premises are:

1) we can't trust ISP with unencrypted DNS query

2) we can trust a third party with DNS query because those are encrypted.

My questions are:

\- Why is 1 true? What are ISP doing with unencrypted DNS queries apart from
knowing browsing habits?

\- With DoH do they not know your browsing habit anymore?

\- If there is a possible mitm attack with unencrypted, would it be fine if
the ISP themselves ran a DoH server?

\- For the 3rd party: why would a 3rd party want to run a dns server for free?
ISP run DNS server but customers pay a bill to them precisely to do things
like this.

in other words, in the long term, wouldn't it be a money sink for Cloudflare
to run a DoH server?

\- Instead of just one DoH endpoint, could browsers take a list of DoH
servers, and picking the best one in terms of p95 latency, with some round-
robbin tries occasionally to verify if anything changes?

This would solve the issue of one of the DoH service "having a bad day". Some
amount of centralization would remain but based on an objective criteria
(latency).

(The article answers some of those questions and/or points out the same
concerns)

------
skissane
Why can't I, as an end-user, have the choice of who I want to trust?

My ISP, Cloudflare, Google, somebody else? Give me a dialog box in which I can
choose whether to use DoH and if so which provider to use, and then I can make
up my own mind over which of those parties I trust the most.

~~~
saurik
I would argue that the original way to do this--where applications used the
system resolver--gave you that option, as the system resolver could trivially
be modified to do whatever you wanted (well, unless you are on iOS; on iOS you
are effectively just screwed, because Apple wants to control the entire
software experience... more on this in a later paragraph <\- edit: and also, I
realized you can also do this on iOS!). glibc, for example, allows you to just
give it arbitrary software plugins to implement the resolver, and so you--the
person who configures your computer--have complete control over not just "who"
you want to trust, but exactly "how" you trust them: you can decide to talk to
Google, Cloudflare, or your ISP, and you can use DNS over HTTPS, DNS over TLS,
unencrypted DNS, or something insane like Active Directory for all anyone
cared.

However, at some point, web browsers decided that they didn't like end users
being able to have this control, as, in practice, no one ever took advantage
of it and they always allowed the operating system to select, by default, "use
unencrypted DNS to the current ISP", which was slow, insecure, and often just
_wrong_ (as ISPs--for at least four reasons I can think of, all but _maybe_
one of which I will assert are "bad"\--would sometimes return incorrect
answers _on purpose_ ); and so, now, the new reality is that software is
coming with embedded DNS resolvers that do exactly and only what the software
developer wanted (which, in a world of not just codesigning but notarization,
increasingly means the user has no ability to have any say in the matter what-
so-ever), because "developers know better than everyone else" (originally,
this was just for performance reasons or to avoid strange behavior from ISPs,
reimplementing the logic, but not the protocol, though now they are using that
control to take it further).

For clarity, I will restate "the premise": if software uses the system
resolver (the one that people erroneously like to claim is somehow synonymous
with "unencrypted DNS to your ISP"), anyone who wants to use DNS over HTTPS
can, AFAIK on _any_ non-iOS operating system (if nothing else, by simply
setting their DNS server to 127.0.0.1, if for some reason you can't just plug
in your own resolver backend; edit: oh, actually, _even on iOS_ , by using a
local Network Extension, which is how Cloudflare's 1.1.1.1 app works), do that
for not just their web browser but for all software on their entire system at
once; and, when some new protocol comes out that is _even better_ than DNS
over HTTPS (maybe by solving some latency issue, or by using a decentralized
resolver, or adding cryptographic verification), it would be equally easy to
change your system resolver, and by extension all software on your entire
computer, at once, to use _that_ protocol.

And now, again, for clarity, I will restate "what happened": that this was
deemed not something that either users or, interestingly, operating systems
should be in control over (though it would be hilarious to see Apple try to
fight this one out from their fortress of iOS, claiming you don't get to be in
the App Store unless you use the system resolver, a similar battle to their
issues with people doing HTTPS requests that don't use their App Transport
Security library); and so, instead, software (possibly ever single piece of
software you use: not just your web browser, but random desktop chat
applications, e-mail clients, games... maybe even network-oriented command
line tools such as curl!) will now embed their own DNS implementations, making
it "your guess is as good as mine" as to what protocol or root of trust any
specific piece of software is relying on :/.

~~~
cremp
Discovered this Fun fact last night; the systemd resolver since V242, used
cloudflare as a fallback DNS. [0] The fallback has been Google and quad9.

In theory (haven't tested it,) this means that even if I give 0 DNS servers on
a DHCP request, and I ignore any DNS requests to the gateway (if I wanted a
machine totally unable to resolve) I can't do that because a distro choice to
use the systemd resolver.

[0]
[https://github.com/systemd/systemd/blob/master/NEWS#L751](https://github.com/systemd/systemd/blob/master/NEWS#L751)

~~~
LinuxBender
This is going to create a lot of noise (firewall drops) in networks where
Linux is expected to be isolated. Many companies do this and many of those
companies will start seeing alerts in their security operations centers. I am
curious how most of them will respond.

~~~
heavyset_go
It's trivial to disable the systemd resolver or change the fallback DNS
servers to something other than Cloudflare.

~~~
LinuxBender
Agreed, but people tend to not customize settings proactively. I usually see
reactive changes. More often than not, people don't even know these changes to
the base OS are occurring.

------
jannes
While I disagree with Mozilla's decision to switch users to centralised DoH, I
believe they are right in their assesment that the current DNS infrastructure
is not very private.

And something like DoH sounds pretty cool.

Without encryption there are a few nefarious things ISPs can do quite easily:

\- transparent DNS proxies

\- DNS query tracking

These things are pretty bad if you're affected, but I believe this shouldn't
be addressed by centralisation of DNS.

~~~
stock_toaster
I think Dns-over-TLS (DoT) is a much better solution. It seems super weird to
me that Mozilla isn't pushing for that, and is instead pushing for
DoH+cloudflare.

------
musicale
> We have to keep in mind that if a DNS lookup is slow, the entire internet
> feels sluggish. Slow DNS = Slow internet.

Ridiculously low DNS TTL is an idiotic curse of the internet, but fortunately
(and currently) one that can be (mostly) fixed locally.

------
tptacek
This is a disinformation campaign. DoH doesn't centralize DNS or DNS privacy.
Anyone can run a DoH resolver server, and it'll do just as good a job as Cloud
Flare's.

The people crusading against DoH favor a different centralized DNS standard,
DoT. DoT is DoH with essentially one difference, which is that it runs on its
own port, so that network operators can block it. There's a sprawling cottage
industry of security and analytics products that rely on passive DNS analysis
to function, and people who sell those products or operate networks that use
them have a strong disincentive to block DoT and require users to speak
standard, plaintext DNS to servers they can monitor.

Note however that once it's up and running, DoT is just as centralized as DoH
is; it has to be, because it provides the exact same service model.
Centralization and competition has nothing to do with their real concerns, or
they'd be lobbying against DoT as well.

I can't say I'm especially fond of Firefox's decision to default to Cloud
Flare for its DoH resolver, because I loathe Cloud Flare. Google Chrome does
something more sensible: it won't change your DNS server at all, but will
upgrade you to DoH if your nameservers support DoH. You should use nameservers
that do, because DoH is valuable: if you're in the US, your ISP is almost
certainly passively monitoring and recording your DNS queries, and you
shouldn't let them. Personal feelings about Cloud Flare aside, you are better
off with them than with your US ISP's DNS servers.

There's a strain of Unix sysadmin criticism against DoH that says it's wrong
for browsers to co-opt DNS resolution at all, and that they should use the
system's resolver. This is nonsense. First, almost no matter which browser you
use, you should trust their DNS resolution code more than your operating
system's; it will be more security-conscious and probably more performance-
conscious code --- DNS resolution maintenance is the job you get on Microsoft
or Apple's OS teams to punish you when you break the build. More importantly,
as anyone who's ever had to mess around with libares will tell you, browsers
have always needed better DNS resolution than libc offers, because they need
fast async lookups.

There's another strain of criticism against DoH that Unix nerds won't see much
of but which is probably even more important to be aware of: there have been
hearings in Congress about how DoH threatens law enforcement by depriving them
of intelligence for investigations. Ultimately, this is of a kind with the DoT
vs. DoH argument: the people lobbying against DoH _simply don 't believe DNS
lookups should be private_. Another "tell" that you're reading one of these
people is when they start talking about how DNS privacy doesn't provide
complete or perfect privacy --- that you shouldn't care if your DNS lookups
are being monitored, because you're leaking so much information elsewhere.
This is bamboozlement. DoH privacy is extraordinarily low-effort, actually
modernizes the DNS protocol in some ways, and addresses a threat millions of
users are directly facing right now.

You should be wary of any outlet that promotes the meme that DoH is somehow
shady.

~~~
cremp
> crusading against DoH favor a different centralized DNS standard, DoT

I think that there are 2 camps against DoH. 1. Default centralization to
specific points. EG firefox using cloudflare first and foremost, nobody else.
2. DoH adding complexity. DoT was (practically) superseeded with DoH anyway,
if for nothing more than adoption.

> criticism against DoH that says it's wrong for browsers to co-opt DNS
> resolution at all, and that they should use the system's resolver. This is
> nonsense.

So... your argument is that I can't trust the OS to 'do the right thing' but I
should trust the browser, because they know best? If you can't trust the OS,
then how could I possibly trust a browser running on the said, untrusted OS?

Honestly asking how you came to that conclusion because the train of trust is
broken on the OS level, so anything above is moot.

> DoH simply don't believe DNS lookups should be private

Problem with DoH is that it _is_ private _up to the endpoint_. Nobody can
listen on the request, but nobody is preventing the endpoint from telling
everyone else that 'Joe Smith visited Youtube at {timestamp} Once the endpoint
has the request, they have your info and can just as easily sell that to
telcos.

End-to-End encryption and privacy is only as good as the people on the other
side. Can't trust Alice with your message, then don't send it.

> modernizes the DNS protocol

If by modernize you mean convolutes. If I have a resolver on my network
serving qwer.localnet. Browser asks cloudflare, returns nxdomain, then my
system resolver asks for it; that is _far_ more latency and is far from the
modern proper solution for speed and privacy. Cloudflare now knows that I have
a local domain, qwer.localnet.

~~~
tptacek
With respect to your OS versus your browser, you're using "trust" in a
different way than I am. Obviously, you have to "trust" your OS in a strict
engineering sense; if the kernel is compromised, nothing the browser can do
will meaningfully mitigate that. That's not what I'm talking about. I'm saying
I "trust" my browser developer to care more about DNS security than I "trust"
my OS vendor, just like I "trust" Chrome's X.509 handling more than I "trust"
the certificate validation code that ships on the OS. It's not that I think
the OS developers are malicious; quite the opposite. I just believe, with some
evidence, that they're not incentivized to adopt modern security mechanisms
for those features.

My operating system ships with IPSEC VPN support. I'm not going to use it,
even though I think highly of the poor souls tasked with maintaining it for
Apple; I use WireGuard instead, because I trust Jason and, more importantly,
Jason's incentives, more than I trust Apple. I certainly don't feel like I
need _permission_ from my OS (in the form of them formally adopting WireGuard)
to do so.

With respect to endpoint security – I assume really what you mean is the
security of the resolver server that you use, which can indeed monitor your
DNS requests – yes! You do have to trust the DoH server with your requests.
You should choose one that you do trust. But because your US ISP is already
violating that trust flagrantly, you're almost better off with any off-network
DoH server you can find. Really, if you're a stickler, you'll just run your
own DoH server. You can't do worse than the status quo ante.

I don't care about the "convolution" argument.

------
iofiiiiiiiii
The article claims that there is some push towards centralized DNS. I do not
see the evidence to support this - the only reason Cloudflare is the default
in Firefox is that DoH implementations are still rare. PowerDNS is free to
implement DoH themselves if they want their customers to be competitive, and
so is every ISP.

~~~
sp332
CF was chosen because they signed a contract with Mozilla to reduce the amount
of data they collect to the bare minimum.

~~~
throw0101a
Can (US) National Security Letters override such contracts?

One nice thing about 'plain' DNS(-over-54; Do53) is that it is distributed in
nature, so to get any kind of massive surveillance you have to visit multiple
ISPs.

~~~
heavyset_go
> _Can (US) National Security Letters override such contracts?_

Yes, and normal, everyday courts can compel companies to siphon their
customers' data and keep it secret that they're doing so, despite any claims
that the companies make in their privacy policies.

------
vtsingaras
The point presented by the post is plain wrong, people were using non-ISP
recursors for a long time already and this was not problem due to the fact
that EDNS Client Subnet has been a thing for a while now.

The problem mentioned in the post is specific to Cloudflare, as they do not
support EDNS Client Subnet anyway, even over plain UDP53

more reading: [https://www.samknows.com/blog/dns-over-https-
performance](https://www.samknows.com/blog/dns-over-https-performance)

~~~
jgrahamc
Mozilla did their own testing of DoH and its effect on performance in
conjunction with Akamai:

Test announcement: [https://blog.mozilla.org/futurereleases/2018/11/27/next-
step...](https://blog.mozilla.org/futurereleases/2018/11/27/next-steps-in-dns-
over-https-testing/) "To do that, we’re working with Akamai to help us
understand more about the performance impact."

First results: [https://blog.mozilla.org/futurereleases/2019/04/02/dns-
over-...](https://blog.mozilla.org/futurereleases/2019/04/02/dns-over-https-
doh-update-recent-testing-results-and-next-steps/) "The results were strong!
Like our previous studies, DoH had minimal impact or clearly improved the
total time it takes to get a response from the resolver and fetch a web page."

Final post: [https://blog.mozilla.org/futurereleases/2019/07/31/dns-
over-...](https://blog.mozilla.org/futurereleases/2019/07/31/dns-over-https-
doh-update-detecting-managed-networks-and-user-choice/) "Furthermore, we found
that web pages that are hosted by Akamai–a content distribution network, or
“CDN”–have similar performance when DoH is enabled."

~~~
gsich
And nowhere is "TCP" mentioned. I suppose the connection just builds up itself
in 0ms time.

Sure if you already have <5ms latency to your DNS, the additional RTT from TCP
and TLS won't matter much. For the rest: they will.

~~~
judge2020
That's valid, but it's also why HTTP/3 is being pushed for 1-RTT and 0-RTT
connections (0-RTT is most likely going to be enabled for their DoH if it
isn't already)

[https://blog.cloudflare.com/even-faster-connection-
establish...](https://blog.cloudflare.com/even-faster-connection-
establishment-with-quic-0-rtt-resumption/)

~~~
throw0101a
And for that you need TLS tickets/cookies, don't you? (So you can resume your
crypto state.) Isn't that a good way to track users?

Instead of a query coming from an IP (probably NATed with IPv4), you can
identify individual browsers AFAICT.

~~~
tialaramex
The DoH server gets to track users if they use a resumption method, yes. As I
understand it Firefox puts extra work in to ensure that it doesn't share the
backend cache between your HTTPS service and the DNS resolver so there's no
risk that if the DNS service and the HTTPS site you're visiting are owned by
the same people they can correlate that, but they would be able to see that
it's the same browser asking for say, camelporn.example and visit-
arabia.example

If you disabled resumption you could get rid of this issue at a cost of one
extra round trip per DNS lookup.

Nobody else gets to track users this way (at least in TLS 1.3 and I'd assume
DoH implementers will use TLS 1.3 since they are of similar vintage) because
the keys aren't re-used so the identifier for the key will just appear to be
random noise each time.

