
Systemd-resolved DNS cache poisoning - comice
http://seclists.org/oss-sec/2014/q4/592
======
josteink
Am I the only one who is getting fed up with everything under the sun having
their own DNS resolver and DNS cache? It's border-lining madness right now.

Before you had a situation like this. ISP: Server and cache. Router: Client,
server and cache. OS: Client and cache.

So if you ever encounter DNS failing, you only had 3 possible places to look
for errors, or maybe not even errors, just stale data that may or may not
correct itself over time. Flush the caches you can and hope for the best.
Rinse and repeat.

But then came the browsers and decided that 3 levels of caching was good
enough. Let's add more which can fail in the name of performance!

Combine this with mobile, mobility and data-roaming and now you not only have
a DNS-cache (based on already tripple-cached data) which often is guaranteed
to be wrong, your browser may actually have cached the wrong DNS-servers, so
when you roam from Wifi to cell-data your precious LAN DNS servers are no
longer available and your browser just looks stupid.

And now systemd adds to this. What the flying _fuck_?

All this nonsense because someone somewhere decided that a nice, shared,
global OS level client and cache wasn't good enough. Handling this complexity
once would just be _too easy_.

Can someone tell me why? Why wasn't it good enough? What was it that prompted
someone to go reimplement all this wrongly, yet again?

Yes this is a rant, but if there are underlying technical reasons for this
nonsense, I'm very, very interested to hear about it.

~~~
ominous_prime
> And now systemd adds to this. What the flying fuck?

While I agree with you that these layers of caching can lead to headaches,
Linux doesn't have a built-in cache (and distros often don't provide one by
default). Systemd would be one way to provide this, instead of other daemons
that have been used in the past, like nscd, dnsmasq or bind. So it's not
adding yet another layer, just providing the layer at the local host.

As for technical reasons, I'm not sure what you want to hear, each layer
implements a cache for performance and/or efficiency. No one throws in a DNS
cache just because they feel like it.

~~~
jacquesm
Linux doesn't need a cache, you can simply run a local DNS server. The kernel
should not do more than it needs to do. (It's bloated enough as it is)

~~~
ominous_prime
I don't understand your comment. a DNS server is typically also a cache, and
this is what systemd was attempting to provide, without the public server
part. It's the same thing that distros were using e.g. dnsmasq for.

What does the kernel have to do with this? No one is proposing that the kernel
should have a DNS cache.

~~~
rdtsc
> this is what systemd was attempting to provide

Why does systemd need to provide it? I already provides enough things.

Are there any bugs in :

[http://www.thekelleys.org.uk/dnsmasq/doc.html](http://www.thekelleys.org.uk/dnsmasq/doc.html)

I had great success using that. Both installing it on my own or run it part of
Ubuntu's base install.

(And the answer of "but it already provides everything + kitchen sink, might
as well provide DNS caching too" would probably not be what most people would
want to hear).

~~~
RexRollman
Because it is the Emacs of init systems.

~~~
parfe
Emacs is the Emacs of init systems

[http://www.informatimago.com/linux/emacs-on-user-mode-
linux....](http://www.informatimago.com/linux/emacs-on-user-mode-linux.html)

------
amluto
I find the design of systemd-resolved to be very strange. It uses dbus to talk
to glibc, and it seems to be a new, from-scratch implementation of a DNS
resolver. To be clear, I don't really think it matters whether systemd-
resolved is under the systemd umbrella, but I do think that the design has a
lot of unnecessary NIH syndrome.

It turns out that there's a very well-specified protocol by which clients can
ask a local cache on their system to answer DNS queries. That protocol is
called DNS :) I don't see why routing something DNS-like over dbus makes any
sense in contrast to doing it using DNS itself on port 53.

Fedora is experimenting with running unbound as a local caching resolver [1].
This gives caching, DNSSEC validation, and all the benefits from the fact that
unbound is probably much better hardened than the average libc or application-
side DNS client implementation.

[1]
[http://fedoraproject.org/wiki/Features/DNSSEC_on_workstation...](http://fedoraproject.org/wiki/Features/DNSSEC_on_workstations)

~~~
digi_owl
Poettering and crew seems to have such a hardon for dbus that they want to get
their own variant into the Linux kernel (kdbus).

Hell, i have recently seen a fedora bug regarding the use of su -. Where
Poettering argued that people should not use su because it broke dbus.

Instead he seemed to advocate that people ssh into 127.0.0.1 to do their thing
with a different account.

------
agwa
This is a perfect example of why the systemd approach of putting a bunch of
disparate components under a single tightly-coupled umbrella is bad
engineering. Generally, projects focused on doing one thing are able to do it
well, while projects that attempt to do many things have a hard time doing any
of them well. Unfortunately, as the steady stream of cache poisoning
vulnerabilities in other DNS servers have demonstrated, implementing DNS
securely is _really hard_. A less-than-perfect attempt at implementing a
caching DNS resolver _will_ be insecure. For this reason, it's a task best
left to a mature project run by people with DNS-specific experience who can
focus on doing that one thing well.

~~~
Someone1234
> This is a perfect example of why the systemd approach of reinventing a bunch
> of existing components under a single tightly-coupled umbrella is a bad
> approach.

Why? This wasn't caused by systemd's complexity or having too much components
interconnect poorly. It was a bug in one of their implementations that could
have equally happened had it been a stand-alone DNS cache resolver.

> Generally, projects focused on doing one thing are able to do it well, while
> projects that attempt to do many things have a hard time doing any of them
> well.

I'd totally buy that if BIND didn't have such a checkered history. I mean with
BIND 9 they scrapped the entire project and started again because it was so
insecure...

Yet they're only attempting to do "one thing" so according to you they should
have done it well.

> Unfortunately, as the steady stream of cache poisoning vulnerabilities in
> other DNS servers have demonstrated, implementing DNS securely is really
> hard.

This line oddly contradicts the line that directly proceeds it.

> For this reason, it's a task best left to a mature project run by people
> with DNS-specific experience who can focus on doing that one thing well.

According to the previous line they aren't doing that one thing well...

~~~
agwa
The reason why a project with a bunch of tightly-coupled disparate components
will have a hard time doing anything well is that an expert who could do a
good job isn't going to spend their time working on an implementation that can
only be used along with a bunch of unrelated baggage. They're going to work on
an independent implementation that can be used by as many people as possible.

Take DJB. He's an expert and djbdns has a stellar security record. He's not
going to spend his time trying to make systemd's resolver secure. Instead
systemd is going to end up repeating all the same mistakes BIND has made. At
least BIND has a substantial head start over systemd-resolved.

~~~
Someone1234
> They're going to work on an independent implementation that can be used by
> as many people as possible.

So they would work on systemd then? Since it is set to have a far larger
customer base in the near and long term than most independent cache resolvers
(just see the list of distro's who have taken it up).

> Instead systemd is going to end up repeating all the same mistakes BIND has
> made. At least BIND has a substantial head start over systemd-resolved.

Potentially. But I don't think you've successfully shown that that is
inherently due to systemd's design, complexity, or componentization. Instead
it is likely due to systemd's relative immaturity of a codebase and that as
discussed previously most cache resolvers have had a checkered history of
security issues (and assuming systemd is "as bad" as everyone else, it will
have security issues).

~~~
protomyth
> So they would work on systemd then? Since it is set to have a far larger
> customer base in the near and long term than most independent cache
> resolvers (just see the list of distro's who have taken it up).

Is the systemd project passionate about DNS? Is the curl team passionate about
protocols and files? Is the opensmtpd team passionate about e-mail?

I think two of the three are yes answers. The great strength of UNIX has
always been people's ability to flock to places where their passion is shared
and improve and learn (institutional knowledge) about their passions.

I don't think systemd will have passionate people for these extra services and
it seems they don't have the institutional knowledge that others have gained
since this is not a new kind of bug.

~~~
Someone1234
As has already been shown through example, having a single unified purpose
does not make something immune (or even resistant) from security issues.

Tons of projects with a single focus have had countless security problems,
some so bad the entire project was completely scrapped.

So the entire argument is based on a flawed premise.

~~~
protomyth
I think history favors projects, companies, and individuals that concentrate
one something as opposed to big entities that try to be all things to
everyone. This is how small companies are able to dislodge larger ones. Focus
is important in anything. The argument is far from flawed, as limited counter-
examples are hard to find.

------
jacquesm
Pretty weird for Systemd to have its own cache and resolver. If you need a
local resolver and a cache simply run a local DNS server rather than to re-
invent the wheel (in a broken way at that). I'm sure they have their reasons
but this is not the unix way.

~~~
kefka
Systemd is just not the Unix Way. It's some shit program that requires to be
run as PID 1, and takes over the system in the forms of an octopus. And, you
need some gods-awful binary program to even read their logfiles. Yes, it's
like the Windows registry.

Even Debian has succumbed to Systemd.

~~~
vezzy-fnord
_Yes, it 's like the Windows registry._

Nope, that would be GSettings/dconf. It's limited to GNOME, thankfully.

~~~
custardcream
Actually systemd is more like DCOM which is much worse.

The registry is fucking marvellous in comparison.

~~~
talideon
Nah, the equivalent of DCOM is dbus.

~~~
custardcream
Which is what systemd is built on. At least on windows COM stays in user space
but no, fuck it, lets use kdbus on Linux...

~~~
talideon
Sure, that's what _built_ on, and that's why the equivalence was incorrect.
You'd might as well say that systemd = Linux.

------
userbinator
Interesting background discussion here for those wondering why systemd has its
own DNS resolver:
[https://news.ycombinator.com/item?id=8203080](https://news.ycombinator.com/item?id=8203080)

~~~
marcosdumay
Or, in other words, since they want to integrate the entire OS into systemd,
they need a DNS resolver there too.

------
vezzy-fnord
The submitter doesn't state affected versions, but judging from systemd's NEWS
file, the resolved component was first introduced in 213 and later dubbed as
"a pretty complete caching DNS and LLMNR stub resolver" in 216:
[http://lwn.net/Articles/609740/](http://lwn.net/Articles/609740/)

Considering most users run on a backported 208-stable IIRC (I know RHEL 7
settled on that), I'm not sure how widespread this is. Still concerning that
they felt to roll their own and not follow RFC guidelines, though.

Considering how plagued with vulnerabilities BIND was, I'd assume DNS is a
hard thing to do.

~~~
agwa
> "a pretty complete caching DNS and LLMNR stub resolver"

That quote is gold. Most of DNS isn't implementing the protocol, but
implementing all the protections against cache poisoning, many of which are
not intuitive.

But as the oss-sec email says: "systemd-resolved does not implement any of the
hardening recommendations of rfc5452."

------
talideon
Of all the components of systemd that doesn't need to exist, resolved is
definitely one of them. Why they're reinventing the wheel rather than allowing
the user to set up something that works and is battle-tested, such as unbound,
is beyond me.

~~~
JeremyNT
I don't necessarily disagree with the heart of your statement, but it's worth
mentioning that systemd does not _require_ one to use resolved. The user is
still "allowed" to run unbound, dnsmasq, etc., and the preferred solution will
be a question for distribution maintainers and users to answer.

I understand concerns about systemd's scope, but most of the "questionable"
functionality is optional.

~~~
agwa
Small comfort. When a project has the incredibly poor judgment (and, frankly,
hubris) to write a caching DNS resolver without even the most basic cache
hardening protections, and then declare it "a pretty complete caching DNS and
LLMNR stub resolver"[1] instead of just leaving it to people who know what
they're doing, it calls into question the rest of the project. This is a major
reason why so many of us are wary of systemd.

[1] [http://lwn.net/Articles/609740/](http://lwn.net/Articles/609740/)

------
SixSigma
Caches are bugs waiting to happen. Rob Pike ‏@rob_pike 21 Mar 2014

By the same author

"There's no such thing as a simple cache bug."

"Caches aren't architecture, they're just optimization."

~~~
tedunangst
This isn't really even a cache bug. It's a bug in a cache, but it's a stupid
bug to have in any DNS implementation, cache or not.

------
4ydx
Systemd... a very large mass of c code being written in a short amount of time
with huge functionality creep. Yeah, this is going to end/begin really well. I
swear all of the people commenting about it being a good thing are not
seasoned developers.

