
Why does 1.1.1.1 not resolve archive.is? - stargrave
https://webapps.stackexchange.com/questions/135222/why-does-1-1-1-1-not-resolve-archive-is/135223#135223
======
amarshall
Previous discussion concerning this, which includes replies from Cloudflare:
[https://news.ycombinator.com/item?id=19828317](https://news.ycombinator.com/item?id=19828317)

~~~
cnst
> We are working with the small number of networks with a higher network/ISP
> density than Cloudflare (e.g., Netflix, Facebook, Google/YouTube) to come up
> with an EDNS IP Subnet alternative that gets them the information they need
> for geolocation targeting without risking user privacy and security. Those
> conversations have been productive and are ongoing. If archive.is has
> suggestions along these lines, we’d be happy to consider them.

I actually forgot to consider this angle, as the discussion was centred at
archive.is. I'd imagine 1.1.1.1 has a very negative consequences for Netflix
local caches, for example. This is where your local homegrown provider gets to
save significant $$$$ due to interconnect/bandwidth costs by hosting a local
Netflix cache through their appliance, and you get to benefit by the latest
shows being locally cached, delivered at maximum speeds, instead of being
hailed through the whole internet each time for each device.

If you're using 1.1.1.1, you're basically not only making sure that your
internet will be much slower due to suboptimal CDN performance by any CDN
other than Cloudflare CDN, but that you're also going to needlessly be running
extra simultaneous streams of your favourite shows instead of fetching a local
copy from your own ISP, increasing the cost of bandwidth transit to your ISP.

And, remember, you don't actually get any extra privacy by bypassing ECS in
the first place, because your exact full IP address will have to be used to
establish any subsequent TCP or UDP connections to make those requests for
actual content in any case. You're basically breaking the whole internet by
using 1.1.1.1, all for no real benefit! It's worse than we initially thought!

~~~
throw0101a
> _I 'd imagine 1.1.1.1 has a very negative consequences for Netflix local
> caches, for example._

Is it possible for Netflix to use anycast?

For their appliances they can advertise to an ISP's routers via (i/e)BGP or
OSPF / IS-IS to keep traffic internal, but have a fallback of having a
presence in various IXPs.

Isn't this how Cloudflare works, anycast?

~~~
GhettoMaestro
Yes Netflix is anycasted at the edge. Has been for years.

~~~
cnst
Just because a given CDN uses anycast doesn't mean that they don't also use
ECS as well. In fact, Cloudflare CEO's own wording seems to suggest that all
of these mentioned providers still need ECS even though they do run anycast.

Basically, it would seem that Cloudflare is trying to close the performance
gap by artificially limiting the performance potential of alternative CDN
providers to match their own levels.

------
nindalf
This link and the two answers within demonstrate something important, broader
than the DNS related issue at hand.

Both make implicit assumptions. One assumes the worst of Cloudflare and thinks
“what’s the worst reason Cloudflare could have for doing this. How do they
profit off this?” And the other assumes that Cloudflare has good intentions.

Neither answer is technically wrong. Both flow logically from their initial
assumptions. But it shows how different our conclusions can be depending on
where our initial biases lie. For the person who believes the first answer and
says “prove to me that Cloudflare isn’t doing something nefarious”, it’s not
possible. The analysis is correct and can’t be challenged unless the initial
assumption is challenged. And for people who _strongly_ believe that
Cloudflare has bad intentions, nothing can be done to change their mind.

In this example it’s Cloudflare but it applies to any person or organisation
that we feel strongly about.

~~~
nabla9
You have to look at Cloudflare user agreement's and texts that describe their
relationship to their customers.
[https://www.cloudflare.com/privacypolicy/](https://www.cloudflare.com/privacypolicy/)
and [https://developers.cloudflare.com/1.1.1.1/commitment-to-
priv...](https://developers.cloudflare.com/1.1.1.1/commitment-to-
privacy/privacy-policy/privacy-policy/)

You can only held companies accountable for the laws and explicit written
promises and legally binding agreements.

Currently the price companies pay for privacy violations is low. If a company
like Cloudflare writes down all the privacy promise in legally bind manner and
puts themselves into legal and financial liability that is above the norm for
breaking the contract intentionally it can increase trust.

Companies can do much more than they do now. They can put explicit bounties
for whistle blowing them and revealing privacy violations. They can hire
trusted third parties to do privacy audits and handle whistle blowing.

~~~
pnako
None of that is legally binding if you don't have a contract with them.

~~~
nabla9
Click-wrap agreement and browse-wrap agreement are both contracts.

[https://en.wikipedia.org/wiki/Browse_wrap](https://en.wikipedia.org/wiki/Browse_wrap)

[https://en.wikipedia.org/wiki/Clickwrap](https://en.wikipedia.org/wiki/Clickwrap)

~~~
cobbzilla
But you must actually “click” or “browse” for it to be enforceable, right?

~~~
nabla9
Not necessarily. There can be implied consent.

Of course, in that case you can't put surprising terms into the agreement if
they are disadvantageous to the user. Courts don't see that a meeting of the
minds took place.
[https://en.wikipedia.org/wiki/Meeting_of_the_minds](https://en.wikipedia.org/wiki/Meeting_of_the_minds)

~~~
cobbzilla
Sure, if you’re actually visiting the site. But (at least in the US) didn’t
the recent LinkedIn case find that if my scraper pulls public data off your
site, the TOS doesn’t apply?

This court decision doesn’t mean “no rules for scrapers”, rather it means
“different rules for scrapers, independent of any site-specific TOS”. Or did I
misunderstand the decision?

~~~
nabla9
Consumer law applies for consumer users and has different protections than
other users have.

Web scraper as a consumer use is hard to argue.

------
profmonocle
> I consider EDNS-less requests from Cloudflare as invalid.

If your site depends on a DNS extension that's only 3.5 years old (and
designed to be optional), I think it's fair to say your site is just offline
for some users due to a config mistake.

You're free to set up your servers however you like, but there's wisdom in
Postel's law.

~~~
Thorrez
Archive.is does not block all requests lacking EDNS. They specifically block
requests coming from Cloudflare's datacenters. Cloudflare is not accidentally
misconfiguring their EDNS, Cloudflare is intentionally not sending EDNS.

~~~
dooglius
Do you have a source for this?

~~~
Godel_unicode
[https://mobile.twitter.com/archiveis/status/1018691421182791...](https://mobile.twitter.com/archiveis/status/1018691421182791680)

~~~
dooglius
I've seen that, it doesn't really clarify whether the block singles out
cloudflare in particular, or whether cloudflare is the only (significant) DNS
resolver that the block happens to affect.

------
tedk-42
I really don't see this as a problem of Cloudflare.

End users switching to Cloudflare's DNS endpoint are doing so because they
feel the DNS provider is both faster and more secure.

They rightly made the decision NOT to pass on the end user's IP information to
the upstream DNS server. I agree with this decision and they are acting in my
best interests in doing so. To draw some kind of nefarious intention from this
is absurd.

Until Cloudflare are proven to be nefarious actors, I'll continue to use their
service.

~~~
oarsinsync
> They rightly made the decision NOT to pass on the end user's IP information
> to the upstream DNS server. I agree with this decision and they are acting
> in my best interests in doing so. To draw some kind of nefarious intention
> from this is absurd.

In this instance, the upstream DNS server and the resultant HTTP server are
operated by the same organisation. Cloudflare have opted to not provide the
/24 (or /56 if IPv6) network that the original DNS request came from, in the
DNS request. Your computer will then provide the /32 (or /128 if IPv6) that
your request is coming from when you connect to the HTTP server.

What privacy win have you gained by Cloudflare not providing that information
in this instance?

~~~
spzb
In this particular case, you're right. But as a general principle DNS is not
necessarily owned by the same organisation as hosts the website.

~~~
oarsinsync
Correct. It's also worth noting that as a general principle, the DNS server
making the request on behalf of the user is hosted in the same network as the
user, and not an external third party.

In this particular case, it's one CDN taking issue with another CDN only. No
other DNS providers appear to be impacted.

------
darklajid
I'm with some of the people on Twitter: It seems weird (to put it mildly) to
just blackhole your own site with no explanation whatsoever to the end-user.
For everyone on 1.1.1.1 archive.is will now be "down" and they're none the
wiser.

Maybe there's a big backstory here, but without context that seems passive-
aggressive and quite random?

~~~
profmonocle
What's especially weird is that they're returning "127.0.0.3" to Cloudflare's
DNS, rather than a DNS SERVFAIL or REFUSED error. On most systems that will
cause a connection refused error or a TCP timeout. I would assume that was a
network issue on their end, not a DNS problem.

~~~
majewsky
SERVFAIL or REFUSED is also not helpful to the end user. They should return
the IP of a host serving a static single-page website explaining the issue.

~~~
zamadatix
REFUSED will trigger a lookup on the next DNS server in the list, which may
not be Cloudflare, instead of guaranteeing the user can't go to the real page.

------
jchw
I am no expert by any means. However, I strongly suspect EDNS is not actually
needed to run a CDN. There’s a lot of approaches to balancing load and
distributing traffic. An example of another approach would be using anycast
IPs.

I’m also surprised that traffic from Cloudflare DNS users caused any
significant problem. Was it really that much traffic?

~~~
profmonocle
> However, I strongly suspect EDNS is not actually needed to run a CDN.

It's not. The proof is that CDNs existed long before edns-client-subnet was
introduced. All it does is allow the CDN's DNS servers to return the most
optimal A/AAAA records for the client. But the worst that should happen
without it is you get sent to a more distant CDN server, and the content loads
more slowly.

The fact that archive.is somehow suffers without this feature (which, btw,
wasn't standardized until 2016) suggests they're doing something really,
really odd. If I were them, I'd focus on making my system more robust, rather
than demanding the rest of the Internet adopt a relatively young, _optional_
DNS extension.

~~~
cnst
Per
[https://serverfault.com/a/560059/110020](https://serverfault.com/a/560059/110020),
Google's 8.8.8.8 has had support for `edns0-client-subnet` since at least
2013, so, even if it's only been standardised in 2016, it's been a de-factor
standard for quite a while, especially in the internet-technology-years.

Here's an interesting thought — if it's so bad for privacy and isn't necessary
for a CDN, does Cloudflare the CDN simply disregard ECS when receiving
requests from DNS.Google, or do they take it into account?

~~~
profmonocle
> even if it's only been standardised in 2016, it's been a de-factor standard
> for quite a while, especially in the internet-technology-years.

If archive.is thinks that Internet standards should be adopted so quickly,
it's weird that they don't support IPv6 considering it's been a standard since
1998!

Obviously I'm kidding, but only kind of. When it comes to insisting on
adopting new standards, edns-client-subnet is a weird hill to die on,
especially considering it was always meant to be optional.

> does Cloudflare the CDN simply disregard ECS when receiving requests from
> DNS.Google, or do they take it into account?

I don't think they have a reason to use it because they use TCP anycast.
Looking at [https://cachecheck.opendns.com/](https://cachecheck.opendns.com/)
they seem to return the same IPs regardless of geography.

~~~
cnst
When you talk about ECS being optional, you also have to keep the context in
mind.

* Yes, if you're running a local resolver for your LAN, or have a website on a single server, of course ECS should be optional.

* If you're running a CDN (and archive.today does), or if you're running a public resolver at 100+ POPs, then, no, ECS is not meant to be optional.

~~~
wopian
"not meant to be optional" is surely a suggestion and not a requirement?

i.e it's not "(...CDN...) then ECS should not be optional"

------
viraptor
> massive mismatch (...) of where DNS and related HTTP requests come from
> causes so many troubles

Does anyone know what they could mean here? I get that having more open
connections and slow requests is not great, but there are popular attacks
people will try against them in this case. They already have to handle
pathologic cases of slow requests, so handling some small number of slower
clients shouldn't be an issue.

Or are they talking about some other problem?

~~~
miyuru
They are taking about Geo load balancing via DNS.[1]

Just try one of the akamai endpoints to test it. (E.g media.steampowered.com)

For me 1.1.1.1 serves akamai singapore IPs, while 8.8.8.8 serves IPs of my
ISPs akamai cache in Sri Lanka.

If your ISP has a bad route to 1.1.1.1, this just gets worse.

[1]
[https://en.wikipedia.org/wiki/GeoDNS](https://en.wikipedia.org/wiki/GeoDNS)

~~~
profmonocle
Blocking a user because the site might load more slowly for them doesn't make
any sense to me. If the user is choosing to use a DNS server that returns sub-
optimal CDN IPs, isn't that their problem?

~~~
oarsinsync
> If the user is choosing to use a DNS server that returns sub-optimal CDN IPs

How many users are explicitly choosing that? How many users are actually
choosing something very different, and this is an unintended consequence of
their choice, that they would otherwise be unaware of if not for this provider
taking a stand?

~~~
dwild
> How many users are explicitly choosing that? How many users are actually
> choosing something very different, and this is an unintended consequence of
> their choice, that they would otherwise be unaware of if not for this
> provider taking a stand?

Not sending anything at all doesn't solve any of this. If a message was shown
explaining the situation, sure, but archive.is solution doesn't answer your
question at all.

------
bastawhiz
A lot of folks here seem to be saying "if you're going to make a DNS query,
you're only going to make a HTTP request," which is simply untrue. Hell, you
can add a HTML tag to your page to prefetch DNS queries. Browsers prefetch DNS
just for hovering your mouse over a link or typing something into your address
bar (without actually navigating). Should some DNS server know your IP address
just because you moved your mouse over a link? IMO, no.

~~~
jiveturkey
I don't understand what side you're taking here.

Please can you rephrase your argument. 100% serious, I'd like to know what
point you're making.

Pre-emptively: because whatever DNS server you are using already knows your IP
address, regardless whether it's the first query for the site itself, or
subsequent queries for site-related additional resources.

~~~
tracker1
If you hover your mouse over an ad linking to sketchy-service.com ... then the
remote dns host for sketchy-service.com now has your IP address.

~~~
jiveturkey
That seems neither here-nor-there for the 1.1.1.1 service.

Doesn't the browser's internal resolver use an external recursive server
(either the host's configured ones or browser-determined ones)? Chrome does,
AFAICT. As opposed to being a recursive resolver itself, it just implements a
caching stub resolver.

The remote DNS host for sketchy-service.com doesn't see your IP address, they
see the recursive server's address.

------
ggm
ECS is not equivalent to 'send the IP' but is revealing.

the fact that I subsequently connect to another place over HTTP or some other
protocol is distinct from telling a DNS authority who is asking a question
about a domain name: the article implies "its the same leakage" but it isn't:
different people get told.

~~~
cnst
What's the actual meaningful difference, though? ECS is limited to a /24
anyways, so, it doesn't even reveal the exact IP address in any case.

~~~
vavrusa
Disclaimer: I work on 1.1.1.1. You might not consider your /24 as personally
identifying, but others might. The original RFC discusses these problems
fairly well
([https://tools.ietf.org/html/rfc7871](https://tools.ietf.org/html/rfc7871),
Privacy notice and privacy considerations). Frank Denis also wrote a good
summary on ECS ([https://00f.net/2013/08/07/edns-client-
subnet/](https://00f.net/2013/08/07/edns-client-subnet/)). There's a multitude
of ways to fix this - use a whitelist of nameservers to send ECS to to avoid
spraying the source prefix everywhere, encrypt the whitelisted connections, or
aggregate the source prefix into a largest covering server scope (e.g. if the
client is in /24 but nameserver serves the same answer for /16, then using any
address in the /16 would do). We're evaluating all of them as there's
different trade-offs (see
[https://blog.mozilla.org/futurereleases/2019/04/02/dns-
over-...](https://blog.mozilla.org/futurereleases/2019/04/02/dns-over-https-
doh-update-recent-testing-results-and-next-steps/)).

~~~
breakingcups
What do you say to the very often heard criticism that the exact IP address
will be leaked the moment the user establishes a TCP connection to the domain
they just looked up?

~~~
vavrusa
Hi, I answered it in another comment below.

------
cm2187
I don’t understand the privacy reason. If I am querying for domain x, why does
it matter that domain x’s DNS servers know what IP I am querying them from? I
am going to hit their web server directly with that very same IP in a few
milliseconds anyway.

~~~
tomatocracy
There are a few reasons. Here are three I can think of off the top of my head:

Many browsers prefetch DNS for links on webpages these days. So it’s entirely
possible and even common that you may query DNS for sites you never visit,
which would indeed be a privacy leak.

Secondly, many sites have their DNS hosted elsewhere so it may not be the same
people you are leaking the information to.

Thirdly, if the DNS query is transmitted to the site’s DNS servers in plain
text (which most DNS is), then despite eSNI etc anyone who has access to the
wire traffic along the route from the DNS proxy to the site’s DNS servers
(which is probably different from the route your own traffic takes to their
servers) can see which site you wanted to access.

------
chaitanya
Does 1.1.1.1 send ECS info to Cloudflare’s own nameservers? More generally,
does 1.1.1.1 in any way treat Cloudflare’s own nameservers in a special way
and send it information that it doesn’t send to others?

If the answer to these questions is no, then Cloudflare’s reasons for blocking
ECS (ie privacy) carry weight. Otherwise no.

~~~
jgrahamc
> Does 1.1.1.1 send ECS info to Cloudflare’s own nameservers?

No

> More generally, does 1.1.1.1 in any way treat Cloudflare’s own nameservers
> in a special way and send it information that it doesn’t send to others?

No

------
PeterStuer
Not sure why this is a link to stackexchange as the second answer is lifted
from the previous HN discussion on the topic

[https://news.ycombinator.com/item?id=19828317](https://news.ycombinator.com/item?id=19828317)

------
varelaz
I think decision of archive.is is very interesting. 1) They attracted a lot of
attention; 2) They showed the way to struggle with Cloudflare business that
abuse their service.

If several bigger CDNs like akamai or softlayer will consider requests from
1.1.1.1 without EDNS as invalid and block them, Clouldflare wouldn't be able
just to say that it's their own problems

------
Santosh83
I'm using Cloudflare's DoH service built into Firefox and archive.is is
resolvable.

~~~
Operyl
I think that's because Firefox falls back to system resolver if something
funky happens. I'm using it strictly over DoH and it isn't working for me
still.

------
known
Is adding "62.192.168.106 archive.is" to /etc/hosts a work around?

~~~
known
I added 217.79.184.91 archive.is to /etc/hosts and did sudo service systemd-
resolved restart

It works;

------
vesche
Roads...? Where we're going, we don't need roads.

------
LeoNatan25
This is the reason I stopped using 1.1.1.1.

~~~
nobodyshere
A very silly one at that given their reasons for not resolving archive.is are
quite rational and on the contrary makes me want to swap google's DNS servers
for theirs.

~~~
tambre
> their reasons for not resolving archive.is

They _do_ solve archive.is. But archive.is's DNS servers have been configured
to return bogus answers to queries from Cloudflare's servers.

~~~
nobodyshere
Not cloudflare servers in particular. They demand EDNS(optional by design)
which cloudflare does not support due to privacy risks.

~~~
jlokier
From
[https://news.ycombinator.com/item?id=21155852](https://news.ycombinator.com/item?id=21155852)

> Archive.is does not block all requests lacking EDNS. They specifically block
> requests coming from Cloudflare's datacenters.

~~~
nobodyshere
Wow, what a bunch of not-so-smart people.

