
A year of Rust and DNS - bluejekyll
http://bluejekyll.github.io/blog/rust/dns/2016/08/21/a-year-of-rust-and-dns.html
======
SomeCallMeTim
Love the idea of reimplementing DNS in Rust. Would love to see more efforts
like this so that we have secure-by-design language implementation of core
security services.

But BIND isn't just failing because "it's written in C", it's failing because
it's written in terrible C. That said, "terrible C" is probably most every C
routine written by someone with less than 10 years of _solid_ low level
experience, so "writing good C code" is not very scalable. There _are_ active,
solid projects with very few security exploits that are written entirely in C.
Nginx comes to mind.

The article makes a brief reference to DJBDNS, which is written in C, but has
suffered zero security exploits [1], and is extremely performant. And it _is_
being used in production, so presumably if it had any exploits they would have
been exposed by now.

But...DJB's code is (usually?) released under a rather unfriendly (though
mostly open) license, and DJBDNS hasn't been updated in some time, so a more
modern project isn't a bad idea. And Rust developers can probably write solid
code with only a few years of programming experience, which makes it easier to
extend without adding security holes on a weekly basis
(coughBINDcoughOPENSSLcough)...

[1]
[http://cr.yp.to/djbdns/guarantee.html](http://cr.yp.to/djbdns/guarantee.html)

~~~
bluejekyll
> The article makes a brief reference to DJBDNS, which is written in C

About 14 years ago I worked on a project that was a fork of qmail. I got to
know DJB's code quite well, to say the least it is awe-inspiring. The level of
understanding and craftsmanship he has in C is honestly something I am certain
I will never achieve. At the same time, it is some of (for me) the most dense
and obtuse code I've had to read. I'm not a fan of loop unrolling, I think
that's a thing for the compiler to optimize personally.

Anyway, in researching before starting this project, I was very aware of
DJBDNS and all of the tools that make up that suite of tools. It is solid,
like a rock. But like a rock, it is also inflexible. If you want to read an
interesting post from DJB, this is excellent one on AXFR/IXFR:
[http://cr.yp.to/djbdns/axfr-notes.html](http://cr.yp.to/djbdns/axfr-
notes.html)

In reading this I realized that we have different goals for DNS. I want DNS to
be flexible and secure, he clearly wants DNS to be secure and hardened. We
have different goals, this is not to say my goals are better or worse, but I
do think they differ fundamentally from that of DJB's.

You can't exploit something that you can't change.

~~~
petra
You might find this interesting: "However, the closest in functionality and
intent is Dan Bernstein’s djbdns, which aims to be a minimalist, highly secure
DNS server. The latest release of djbdns, including various support tools, is
about 10,000 lines of C as measured by sloccount. We expect that it is
possible to build a feature-par version with Nail that is an order of
magnitude smaller and intend to do so."

[https://people.csail.mit.edu/nickolai/papers/bangert-nail-
la...](https://people.csail.mit.edu/nickolai/papers/bangert-nail-langsec.pdf)

Nail, btw is parser written in the spirit of "language theoretic security",
which is a poweful new way of looking at security problems.

------
jcranmer
I don't know if you're aware of this or not, but when implementing standards
with dozens of haphazard extensions, often the best way to keep track of all
of the extensions is via the IANA assignments (see
[http://www.iana.org/protocols](http://www.iana.org/protocols)). For DNS, the
main page seems to be here: [http://www.iana.org/assignments/dns-
parameters/dns-parameter...](http://www.iana.org/assignments/dns-
parameters/dns-parameters.xhtml)

~~~
bluejekyll
Yes, I am very familiar with that page, but thanks for posting it! I've used
it for double checking many of the record type assignments and the current
sets of crypto graphic assignments.

------
kenOfYugen
An inspirational side-project.

> I want to get a DNS fuzzer running against it to really pound on it, and
> then get some benchmark and comparison tests against other servers.

Even if the benchmarks turn out to be not in favor of Trust-DNS, the added
safety overhead (if any) should be worth it.

> (I’ll try to post more regularly on progress)

Please do. I thoroughly enjoyed the content as well as the writing style!

~~~
bluejekyll
>Please do. I thoroughly enjoyed the content as well as the writing style!

Thanks!

------
nickpsecurity
With Ironsides, that makes at least two of you using safest languages you can
find to try to improve DNS. Good goal and tool to pick. :)

[http://ironsides.martincarlisle.com/](http://ironsides.martincarlisle.com/)

"Take a look at the full list sometime, I think roughly 50% of those could
have been avoided by using (safe) Rust."

This is true for most vulnerabilities I see in C-related apps. We also know
there's techniques to prevent that with acceptable performance. So, outside
hobbyist or non-critical stuff, I tell people to use a different language to
get the baseline of quality/security up. DNS is kind of example that makes it
more true.

Now, I haven't learned Rust yet, so I won't be able to fully appreciate the
article or spot coding ideas until I do. What I did spot was that you slogged
through all the RFC's implementing and testing what you thought should be in
the DNS system. Aside from a DNS server/client, I think one of most valuable
things you could do in this project is create a _single_ specification of
various things in DNS that cites parts of RFC's or advice from real-world
implementations to justify each part. Kind of a walkthrough for other people
creating DNS's that gives them high-level view and/or drill-down into details
of something.

"While in the pit of dispair" "Then I climbed saviors peak"

Haha. Good job getting through all the work as most people quit that I can
tell.

"It’s currently not used in production (as far as I know). I’ve put a lot of
work into validating correctness of what is going on"

It's good you have gone through the specs and have plenty of features. You did
hit on a risk area that might need to be in this section: difference between
the specs and real-world implementations that you _have_ to work with. The
differences could cause your server to fail. If those exist in DNS, too, there
might already be write-ups on common ones out there. Biggest way to find them,
though, is running combo's of popular clients and servers against each other
in testing. So, that's a possible consideration for the future.

~~~
bluejekyll
> With Ironsides, that makes at least two of you using safest languages you
> can find to try to improve DNS

I looked at that a while ago, I should take another. Thanks for pointing that
out.

> I think one of most valuable things you could do in this project is create a
> single specification of various things in DNS that cites parts of RFC's or
> advice from real-world implementations to justify each part. Kind of a
> walkthrough for other people creating DNS's that gives them high-level view
> and/or drill-down into details of something.

I was thinking of doing something like this, but you have an even better idea
here. Actual references back to the code. I might start trying to do that.
(Though that's probably as much work, if not more, as writing the software ;)

> Biggest way to find them, though, is running combo's of popular clients and
> servers against each other in testing. So, that's a possible consideration
> for the future.

Yeah, I've wanted to stand up some automated tests with the most popular
systems out there, but there's a time issue here. If others are inspired and
want to join the project, I'd love help in this area!

~~~
e12e
> I was thinking of doing something like this, but you have an even better
> idea here. Actual references back to the code. I might start trying to do
> that.

Literate programming (in the Knuth sense): yay!

> (Though that's probably as much work, if not more, as writing the software
> ;)

Yep. Hence one of the few big projects I'm aware of that use lp (apart from
Knuth) is:

Axiom:

[http://axiom.axiom-developer.org/axiom-
website/books.html](http://axiom.axiom-developer.org/axiom-website/books.html)

Ed: from the post: > (And writing this post helped uncover a bug, so even if
no one reads this, it was worth it).

Literate programming: yay!

Or perhaps narrative-driven design: ndd, to go with tdd and bdd...

------
dochtman
As a _ring_ [1] contributor, would be curious to hear if it could satisfy your
crypto requirements. Using it could maybe help with your "(oh-my-dear-god I
can not unsee what I saw in there, the C, not Rust)" experience.

[1] [https://github.com/briansmith/ring](https://github.com/briansmith/ring)

~~~
bluejekyll
Funny you should ask, I was just looking at that last week to try and answer
that question.

I was also thinking of getting my feet wet in ring with some of the _easy_
issues on the github page.

~~~
dochtman
Sounds awesome, looking forward to it!

If you are an IRC kind of person, maybe also consider joining the #rust-crypto
channel on irc.mozilla.org.

------
viraptor
bluejekyll, have you thought about splitting the server into separate crates?
(for example the parsing, the dnssec validation, the forwarding/resolving
part, etc.) I'm planning to write a purely forwarding/reporting DNS server
with white/black-lists and it looks like I could reuse 99% of code from you.
But unless I'm missing something, the whole of the server is in one
crate/repo. I did see that it's split into one lib and one bin, but haven't
looked into how reusable that lib is yet.

~~~
bluejekyll
I have, and specifically for purposes like this. Honestly, it just hasn't been
a high priority though.

At least for the client/server code, I think you'd end up with 90% of all the
same dependencies, so I don't know exactly how I'd divide it up. If you have
ideas, definitely let me know!

------
voltagex_
Great article.

>Implementing rfc1035 was deceivingly easy

This stuck out to me - RFC1035 is the first RFC I've read all the way through
and had a shot at implementing. Then you realise it's RFCs all the way down
and there's no easy way to navigate through all the overridden / deprecated
parts.

~~~
nly
Inded. The old RFCs are just the tip of the iceberg, and in the end you have
to pretty much ask "What would BIND do?".

Zone master files are a great example of this. There's no official grammar and
so there are subtle differences in the way different DNS servers parse them.

Its also easy to under-estimate and not notice for example, until you've
written a lexer or two, that the grammar is context sensitive and ambiguous...
so I'd almost guarantee that trust-dns can be made to interpret one
differently to BIND, Knot, or unbound.

------
DigitalJack
Just curious. Is this the normal way:

    
    
        let recursion_available = (0b1000_0000 & r_z_ad_cd_rcod) == 0b1000_0000
    

That just seems redundant. If you declared the var as a Boolean, would the ==
0bxxxx part have been necessary?

~~~
chrismorgan
If r_z_ad_cd_rcod is of type u8, then 0b1000_0000 & r_z_ad_cd_rcod is also of
type u8, containing the value 0 or 128. It is (by design) not possible to
simply cast a number to a bool in Rust; comparing it with some value is the
way you are required to do this. You can do this with `== 0b1000_0000` if you
want, or `!= 0` would do just as well.

Rust is deliberately explicit about conversions between types; it’s less
error-prone, though often more verbose. It’s a part of the language’s
philosophy.

~~~
cpeterso
This seems like a good use case for a bit mask query function. I see Rust has
a bitflags crate:

[https://doc.rust-lang.org/bitflags/bitflags/index.html](https://doc.rust-
lang.org/bitflags/bitflags/index.html)

~~~
bluejekyll
I found that after I had already written a lot of that code. I haven't had
much reason to go back and change it, but any PR's if someone wants to do it,
would be accepted.

~~~
vog
What about simply replacing all "== 0b..." with "!= 0"?

That would already be a good improvement, both in terms of DRY and
readability, without the need of adding another dependency.

~~~
sedatk
It wouldn't be the same thing. For instance x & 0b11 != 0 can return true for
multiple values of x, 0b10 and 0b01 respectively.

~~~
vog
I see. You are sometimes checking for multiple bits at once. Didn't notice
that.

------
fulafel
See also BIND 10:
[https://ripe68.ripe.net/presentations/208-The_Decline_and_Fa...](https://ripe68.ripe.net/presentations/208-The_Decline_and_Fall_of_BIND_10.pdf)

------
nialv7
Nit: Rust doesn't prevent memory leak.

~~~
steveklabnik
It is unfortunate you're downvoted for making a true statement. It's extremely
important that people don't see Rust as a panacea.

~~~
topspin
Fortunately it appears this perfectly legitimate comment has been salvaged.
Steve, I was disappointed that this topic[1] received no discussion at all at
HN. Has anyone inside Mozilla looked at it? The title is a bit click-baity for
academic work, in my opinion, but I do think the points raised are worthy.
Much like recent criticism of PostgreSQL by Uber, one hopes the purveyors of
Rust are also open and respond well to constructive criticism.

1\.
[https://news.ycombinator.com/item?id=12258957](https://news.ycombinator.com/item?id=12258957)

~~~
steveklabnik
It wasn't discussed a lot because it's quite old, and we talked about it a lot
when it was published. I actually met (and then became friends with) Amit in
real life because of this paper; we're both in New York and got coffee to
discuss it.

(It's been a while, but IIRC the issues they had were more about not
understanding what exacty was available and how to use them, at the time. The
project has been going really well since then; Amit has been giving a number
of cool talks about it at various Rust user groups, and they recently started
their own "This week in", though it's called "Talking Tock" because that's a
better name, ha!)

------
signa11
with erlang you can parse the dns-header like so:

,----

| %% extract dns-header fields from a raw packet.

| parse_dns_header(raw_packet) ->

| <<

| ID:16,

| QR:1, OPCODE:4, AA:1, TC:1, RD:1, RA:1, Z:3, RCODE:4,

| QDCOUNT:16,

| ANCOUNT:16,

| NSCOUNT:16, ARCOUNT:16,

| Tail/binary

| >> = raw_packet,

|

| {#dns_header_record {

| id = ID,

| qr = QR, opcode = OPCODE, aa = AA, tc = TC, rd = RD, ra = RA, z = Z, rcode =
RCODE,

| qdcount = QDCOUNT,

| ancount = ANCOUNT,

| nscount = NSCOUNT,

| arcount = ARCOUNT

| }, Tail}.

|

`----

almost verbatim 'transliteration' of sec:4.1.1 of 1035 :)

~~~
masklinn
FWIW you can get code-formatting by prefixing lines with a 4-space indent:

    
    
        parse_dns_header(raw_packet) ->
          <<
              ID:16,
              QR:1, OPCODE:4, AA:1, TC:1, RD:1, RA:1, Z:3, RCODE:4,
              QDCOUNT:16,
              ANCOUNT:16,
              NSCOUNT:16, ARCOUNT:16,
              Tail/binary
          >> = raw_packet,
    

Also for readers, this does depend on customisable (per-segment/field)
defaults: `ID:16` is a shorthand for `ID:16/big-unsigned-integer-unit:1` aka
"16-bits wide segment parsed as an unsigned integer in big endian"

~~~
signa11
> FWIW you can get code-formatting by prefixing lines with a 4-space indent

ah thank you ! that is very good to know.

will it be possible to have some formatting guidelines / examples on the site
somewhere ? similar to what we see on reddit ?

~~~
masklinn
> will it be possible to have some formatting guidelines / examples on the
> site somewhere ?

[https://news.ycombinator.com/formatdoc](https://news.ycombinator.com/formatdoc)

Sadly HN has jack shit formatting-wise (the most annoying part being you can't
escape asterisks so half the time it's going to emphasise a comment section
and remove your asterisks despite that not being what you wanted)

~~~
signa11
> Sadly HN has jack shit formatting-wise

it _is_ 'spartan' :)

~~~
masklinn
I'd be fine with spartan (only support for code blocks) or more featureful
(escapes at least), but as things are it's less spartan and more annoyingly
half-assed.

------
colemickens
Great post, great library. I used it in a CloudFlare dynamic dns client I
wrote to play around with Rust. I'm still very new to Rust so these sorts of
post are really helpful. Thank you bluejekyll!

------
jedisct1
Another DNS-related project in Rust: EdgeDNS
[https://github.com/jedisct1/edgedns](https://github.com/jedisct1/edgedns)

------
pjmlp
Any benchmarks against the MirageOS TCP/IP stack?

It would be interesting to see how the way Rust manages memory and its current
optimizer match against OCaml in a production TCP/IP stack.

~~~
bluejekyll
I'd love to get some benchmarks going. It's not as high a priority for me
right now as getting to feature complete (my definition of that anyway).

The question you pose though is awkward. MirageOS is a unikernel, Trust-DNS is
an application, a consumer of the TCP stack.

I have seen Rust apps built into rumprun, but as of right now this isn't
something on my list, in theory it's possible.

~~~
pjmlp
It is not awkward at all.

You can build a DNS server with MirageOS and it is used like that in the
context to the new Docker versions for Mac OS X and Windows, which is one
example of being used in production.

------
bsder
The only technical comment I have is that it continues to be amazing to me
that destructuring binary data is so stupidly verbose in so many languages. I
think C and Erlang are the only two languages that got this right.

~~~
masklinn
Erlang I can easily see (destructuring binary data in erlang is an absolute
pleasure) but C? Most binary destructuring in C seems to be done with shifts
and masks, there's some gain from implicit integral conversions but I don't
remember it being "right". IIRC the memory layout of bitfields is not
specified so you can't use them to portably destructure binary streams, only
to define "packed" structures for memory saving.

~~~
Jtsummers
> IIRC the memory layout of bitfields is not specified so you can't use them
> to portably destructure binary streams, only to define "packed" structures
> for memory saving.

This is correct as my colleagues have recently relearned. Moving from one
endianness to another seriously complicated their lives for a few weeks (time
to initial discovery of error, time to add in new definitions, time to test
the updated system).

------
arunmu
Would love to repeat this sometime using pure modern C++.

~~~
detaro
It would be really interesting to have a set of extremely similar programs in,
let's say, "matured C", modern C++ and Rust for comparison purposes.

~~~
arunmu
Yeah. That would be really nice as these languages are competing for the same
ground. Go beats all in ease of programming but something is not quite
satisfying with it(my personal opinion). I really like having control of my
memory. But that doesn't matter usually for most of the application processes.

------
masgui
much more fun to write in scala: [https://github.com/MasseGuillaume/Scala-
DNS/tree/master/src/...](https://github.com/MasseGuillaume/Scala-
DNS/tree/master/src/main/scala/dns)

------
killbrad
Isn't trying to write a secure implementation of DNS as it is today, like
adding a chain to your door? Mostly for show?

~~~
johncolanduoni
Security of DNS the protocol (and the way it's deployed) is different from not
exposing the computer running your DNS server to attacks. This is trying to
solve the latter, which seems like a good idea.

