
RFC8482 – Saying Goodbye to ANY - jgrahamc
https://blog.cloudflare.com/rfc8482-saying-goodbye-to-any/
======
anyfoo
(Title of the submission should probably capitalize “Any” as “ANY”.)

I liked the ANY type as a convenience and debugging tool, but I guess the
disadvantages far outweigh that perceived convenience. And now that I read
about the actual semantics of ANY with respect to caches, turns out I was
using it wrong anyway!

~~~
fanf2
In recent versions of BIND ANY queries still work pretty well as a debugging
tool when `minimal-any` is turned on, because you get the traditional full-fat
response over TCP, and dig automatically uses TCP for ANY queries.

I implemented `minimal-any` in BIND to reduce problems caused by large
responses in DDoS attacks. It helps in situations when RRL is not quite
enough.

~~~
anyfoo
Awesome, thanks a lot for that! I was indeed mostly using ANY through dig (as
do most, I suspect), glad to know there's still a way to make it work!

------
jdashg
This is great. It's so important to try to push for improvements externally,
rather than just sucking it up and implementing onerous workarounds.

It doesn't always work out, but this sounds like a good and reasonable outcome
this time. (Though it's their blog, so of course it sounds that way!)

------
derefr
Even though I know how it works, and why it works, I'm still kind of surprised
that individual DNS records have individual TTLs. It feels unfamiliar/alien,
compared to all the other hierarchy-of-caches systems I work with.

Every intuition I have about caching would lead me to imagine an alternative
implementation of DNS where zones are documents, and intermediate resolvers
retrieve and cache entire zone documents from their canonical hosts, in order
to respond to queries for individual records from stub resolvers.

Is there some big use-case for the DNS records within a zone to have different
TTLs? Would it really be so inefficient if the zone as a whole just had one
TTL, which was set as the minimum of the TTLs required by any of its records?

Does DNS not have spacial locality to its access patterns? I.e., if an
intermediate resolver retrieves the A records for a domain, does that not
imply that it'll soon also want the MX records for that domain, and the TXT
SPF records, and—essentially—all the records with some probability? Is there
no justification for pre-fetching these?

And, if both of those statements were to be true—that DNS could get by just
fine with one TTL per zone, and that DNS queries do have a spacially-local
access pattern—then why would you want to design DNS in a way where
intermediate resolvers retrieve individual records, rather than entire zone
documents? Would a modern ground-up DNS architecture do things this way?

(I have the suspicion that the answer lies in security-through-obscurity, vis.
the reason AXFRs are prohibited. So, as an addendum: I ask the same questions
as above, but this time presuming your DNS daemon has Row-Based Access Control
(RBAC), such that an intermediate resolver retrieving a zone document would
only see the records its upstream resolver thinks that particular client
should be allowed to see—e.g. enterprise Intranet clients get everything,
while public Internet clients get a trimmed view. This implies secure DNS, but
just take that as a given.)

~~~
kccqzy
> Does DNS not have spacial locality to its access patterns? I.e., if an
> intermediate resolver retrieves the A records for a domain, does that not
> imply that it'll soon also want the MX records for that domain, and the TXT
> SPF records, and—essentially—all the records with some probability? Is there
> no justification for pre-fetching these?

Probably not. If I want to visit your website, I might just need A and AAAA.
That doesn't mean I will soon want to send you emails (MX) or look at your
google site verification token (TXT).

> Is there some big use-case for the DNS records within a zone to have
> different TTLs? Would it really be so inefficient if the zone as a whole
> just had one TTL, which was set as the minimum of the TTLs required by any
> of its records?

Definitely no. I tend to set low TTLs for A and AAAA records because in
practice they do change very often. Depending on your setup, it possibly
changes every deployment. But on the other hand MX is long-lived and I set it
to many days. This is important security implications: if say, Gmail, has
cached my MX records for many days, it will be very difficult for a DNS
takeover attack to maliciously redirect my mail or other sort of DNS poisoning
to do the same.

~~~
emmelaich
Fun fact: mailers will use the A record if a domain doesn't have an MX.

See RFC 5321

------
mehrdadn
Could someone explain why standards are "RFC"s? Can't anyone request comments
for something? Why would a request for comments be treated as a standard?

~~~
pas
not all RFCs are standards. RFCs that become one are labeled as STD and a
number, but nobody uses those (afaik)

[https://www.ietf.org/rfc/std-index.txt](https://www.ietf.org/rfc/std-
index.txt)

[https://en.m.wikipedia.org/wiki/Internet_Standard](https://en.m.wikipedia.org/wiki/Internet_Standard)

------
kazinator
Why would it be harder to implement then simply iterating over the known types
and catenating the results of those queries.

Say we have A, CNAME and MX type records in our system. If we see ANY, let's
convert it up-front into into three queries: A, CNAME and MX. Combine the
results and that's it.

ELIA5

~~~
geofft
Here's why: suppose some types are easier to answer than others. Maybe A and
CNAME and AAAA are cached, MX is in the database but not cached, and NSEC3
requires computing a result. In order to correctly implement ANY (pre-
RFC8482), you must convert it into _all_ of those, and wait for the slowest
one. An implementation like Firefox that asks for ANY because it just wants A
and AAAA is wasting both the client and the server's time.

However, also, pre-RFC8482 that's not what ANY is defined to mean. A non-
authoritative caching resolver is permitted to respond with _only the records
it has cached_ if something is in cache. So you're already getting a random
subset of records. So the behavior is unhelpful for clients and also unhelpful
for authoritative servers, and not in a way where the unhelpfulness trades off
for helpfulness elsewhere.

~~~
kazinator
That semantics is indeed useless.

And to begin with, applications should never be interested in all record
pertaining to a domain; they want specific records. If I'm sending mail, I
want a MX record for a domain. I don't want the SPF record unless I'm
receiving mail which implicates that domain. If I'm making a HTTP connection,
I don't want any of these; I want an IP address. So _no_ semantics of ANY is
easily justifiable.

On the other hand, we can't claim that this semantics is hard to implement.

~~~
geofft
It's hard to implement _in a way that makes sense in the constraints of the
world as it is_ (people asking for ANY when they have some meaning in mind
other than the nonsensical specs) and especially in a way that doesn't enable
DNS reflection attacks. In an ideal world, sure, you could enable ANY provided
nobody either good or evil ever used it. And in fact Cloudflare says they did
previously implement it just fine, by littering if statements everywhere.

------
LinuxBender
I am happy to see it go. For years I've had to block it in iptables using:

    
    
        iptables -t raw -I PREROUTING -i eth0 -p udp -m udp --dport 53 -m string --hex-string "|0000ff0001|" --algo bm --from 40 --to 65535 -j DROP
    

Most bots are poorly coded, so most of them don't try the individual records
when ANY times out. ANY is also a decent generalized amplification attack
vector. There are plenty of other ways to do amplification attacks, but this
one is quite brain dead.

Thankyou to the folks at CloudFlare for submitting this.

------
jedisct1
But... Qmail! :)

~~~
wereHamster
What does qmail use it for?

~~~
majke
Qmail used to do ANY.

Qmail, as opposed to other software, was sane enough to understand that lack
of MX in ANY didn't mean it wasn't there. It meant it wasn't in a cache, so it
would retry with just MX.

Our initial proposal (to do REFUSED for ANY) indeed had a chance to break some
older qmail installations. This is why we engaged in a longer process and
found acceptable solution - HINFO. HINFO is both backwards compatible (qmail
will work fine) and solves our problems with ANY. Win-win.

~~~
eridius
Why did it do ANY?

~~~
msbarnett
> This is really just an old workaround that's no-longer needed. Let me
> explain...

> Once upon a time, back in 1996, there was a really unfortunate bug in the
> most popular DNS server software (BIND 4.9.3): it did not respond correctly
> to "CNAME" requests (that is to say, requests for any CNAME data about a
> particular domain name). This is critical information that an email server
> needs to know to do its job. Thankfully, there was a way to work around the
> problem: "ANY" requests. These requests ask the DNS server, essentially, for
> any and ALL information it has about the domain name in question, including
> CNAME information.

> These ANY queries have two big problems:

> As you might imagine, for big domains with lots of mirrors (e.g. gmail.com),
> that's a lot of information, and so the response can be quite big. Big
> responses pose two problems: first, it's a waste of bandwidth, and second,
> it can expose a bug in qmail's handling of large DNS responses (see the next
> patch).

> ANY queries are often not cached by relaying DNS proxies (for whatever
> reason), and so ANY queries cause more traffic EVEN behind a caching DNS
> proxy server.

------
dboreham
Holy crap I remember when RFCs had three digits!

------
superkuh
> This got us thinking: do we really need it? "ANY" is not a popular query
> type - no legitimate software uses it (with the notable exception of qmail).

Cloudflare once again pushes for broad changes to the way the net operates
that benefits themselves and other centralized corporate players without any
benefit to individuals that actually use the net in the sense of using the
internet and not just a web browser.

The kind of exploration and direct learning that was possible when I was a kid
growing up 90s/00s is slowly being phased out as the money seeps in.

~~~
zamadatix
What exactly about the deprecation of ANY do you disagree with (other than
Cloudflare proposed it and it used to exist)? Also the blog post and RFC seem
to make convincing arguments for the general health of the internet, do you
disagree with these and if so why?

~~~
superkuh
I've always used it to explore the internet for fun as a person without any
profit motive.

Cloudflare's argument's are all based around ANY making it slightly harder for
them to make a profit. A small chance of DNS amplification attacks in DNS is
what Cloudflare thinks about because it's their business. But there's no
reason to believe this is more important than individual users wishing to see
what servers are behind a domain.

Combined with GDPR killing off whois the internet is a much more boring, less
transparent place.

~~~
j0057
Limiting ANY queries to just TCP would have been a reasonable middle ground,
but now Cloudflare has a cleaner codebase and does not have to do any
difficult thinking around what to do with synthetic rrsets in response to ANY
queries. Good for Cloudflare!

~~~
majke
> Limiting ANY queries to just TCP would have been a reasonable middle

This was proposed solution, the problem is that in case of attack, against a
valid Authoritative service, launched via open resolvers, the open resolvers
would just download Gigabits of ANY traffic with TCP. Read about this here:
[https://fanf.livejournal.com/140566.html](https://fanf.livejournal.com/140566.html)

------
drywater
I’m not convinced that ANY is that bad. They wrote some bad code, they have
special goals and all of a sudden ANY is not good for you.

~~~
Someone1234
People are using it for reflected DDoS attacks. This has resulted in DNS
providers scoping ANY to the point of breaking it. So even without removing
it, it was horribly broken and returned inconsistent results.

In essence all this change does is remove the fiction of ANY, with or without
RFC8482 ANY wasn't reliable enough for real usage.

~~~
deathanatos
Given how bulky ANY is, and the fact that we're changing the standard anyway,
would not just restricting ANY to TCP queries "fix" the DDoS issue? The
attempted TCP connection would not be formed (the victim would RST it).

(I agree with other issues pointed out by the article, and there are other
reasons why, as a RR type, I would still axe ANY. But the functionality of
being able to query all RRs on a server is often useful for debugging, though
I think there are other practical ways to work around that. (Issue a query for
many common RR types.))

