
Saved 10 billion DNS queries per month by disabling DNS Prefetching - necro
http://www.pinkbike.com/news/DNS-Prefetching-implications.html
======
Xk
That's all very nice, but it seems to do more with the fact that each person
gets his/her own subdomain. And given that I found three xss's in about five
minutes [edit: and then like ten more in the next five seconds, after
realizing any input box works] doesn't give me confidence in their abilities.

[http://www.pinkbike.com/news/search/?q=%3C%2Ftitle%3E%3Cscri...](http://www.pinkbike.com/news/search/?q=%3C%2Ftitle%3E%3Cscript%3Ealert%28%22xss%22%29%3C%2Fscript%3E&x=0&y=0)

[http://www.pinkbike.com/product/compare/?items=466,%22%3E%3C...](http://www.pinkbike.com/product/compare/?items=466,%22%3E%3Cscript%3Ealert%28%22xss%22%29%3C/script%3E)

[http://www.pinkbike.com/photo/list/?date=all&text=%3C/ti...](http://www.pinkbike.com/photo/list/?date=all&text=%3C/title%3E%3Cscript%3Ealert%28%22xss%22%29%3C/script%3E)

[http://www.pinkbike.com/buysell/list/?q=%3Cscript%3Ealert%28...](http://www.pinkbike.com/buysell/list/?q=%3Cscript%3Ealert%28%22xss%22%29%3C%2Fscript%3E&category=0&pmin=&pmax=&region=3)

[http://www.pinkbike.com/forum/search/?q=%3C/title%3E%3Cscrip...](http://www.pinkbike.com/forum/search/?q=%3C/title%3E%3Cscript%3Ealert%28%22xss%22%29%3B%3C%2Fscript%3E)

Edit: I've stopped adding xss's. It's actually harder to find input boxes
which _don't_ lead to xss's than ones which do.

~~~
Xk
Whoever it was that fixed the XSS (necro?), I'm impressed how fast that was.

But please don't rely on just escaping < and >. You have to worry about
double-quotes too, I can end a string and add a "onload" or "onfocus"
attribute if it's already in a tag. And sometimes you have to worry about
single quotes. In fact, there's a lot to take a look at.

Instead of just fixing the case at hand, try to be proactive about it. Check
to make sure you don't have anything else.

Edit: Click the search box, for example.
[http://www.pinkbike.com/forum/search/?q=%22%20onclick=%22ale...](http://www.pinkbike.com/forum/search/?q=%22%20onclick=%22alert%28%27xss%27%29%22%20foo=%22)

~~~
necro
Yup. I actually wrote a filter to our framework when it was built years ago,
but as new people help out on dev they don't always use the framework. My bad
and I do appreciate the smack. I humbly bow to you with egg on my face.

------
ars
Summary (he really should have said this part upfront):

He is (ab)using subdomains, by giving every user a different subdomain on his
site. So a typical page can have links to hundreds of different subdomains on
the page.

For a normal site disabling prefetching is not necessary or a good idea.

This article should be renamed: "Why using hundreds of unique subdomains is
not a good idea." Besides all the DNS queries, you are also messing with
caches.

I don't know anything about his site, but I don't see any reason that every
user needs a unique subdomain.

~~~
bodyfour
One reason to do per-user domains is to prevent cookie leakage caused by user-
specified content. A few years ago LiveJournal got bitten by a firefox bug
that allowed JS execution inside CSS. Since they allowed each user to style
their own page that meant that a malicious user could make the CSS at
"www.livejournal.com/~evil_user" capture the viewer's www.livejournal.com
cookies. That's why they moved everybody to username.livejournal.com, which
previously had been a pay-only feature.

I think the really perplexing about this article is why browsers are so
aggressive with DNS prefetching. If there are links on the page to 300
different domains, is there much benefit to precaching ALL of them? It seems
clear that very few of the domains are likely to be clicked on. If anything
they're probably hurting performance by bombarding the local nameserver with a
flood of requests.

It sounds like browsers need a better precaching heuristic.

~~~
buro9
That doesn't sound like a good enough reason, why not just use HTTPOnly
cookies to prevent that?

~~~
bodyfour
Well, first off the 2006 LJ issue predates the implementation of HttpOnly
cookies I think. They're a fairly recent invention.

HttpOnly doesn't really fix the issue though. It prevents evil js from
straight-out stealing the cookie, but it can still make HTTP requests that
will include the cookies. Therefore the script could still do bad things the
user's behalf (add or remove friends, scrape private data, post spam, etc...)

The LJ story is just an example of a general lesson: if you can put user-
generated content in a different domain (i.e. browser security context) than
your authentication-critical cookies you are in a better position. That way
any javascript leakage at evil-user.yourdomain.com can't harm your users any
more than if they were viewing it at www.evil-users-domain.com.

------
davidu
The real issue is that the cost per query is negligible here and DNS providers
shouldn't charge by the query yet almost _all_ of them do. They have to have
some metric to separate the big guys from the little guys and charge the big
guys more, but this is a crude way to do it.

Consider that publishers often have little control of how many DNS requests
they get, so to charge for something out of your control seems utterly bizarre
to me. Nice to see in this instance, publisher was able to make a meaningful
improvement.

Also keep in mind, I used to run the largest free DNS service in the world so
I'm well aware of what I'm talking about and am totally biased on these
matters. :-)

~~~
necro
The thing to note is that implementation of browser prefetching is further
putting DNS requests out of the control of the publishers. 10 billion queries
was something like 5000 dns req/s and that is considerable resources of some
dns service. I wish dns was not per query costs, but it would seem that is the
metric that resources would be based on. The point of the article is to let
people know that if you fall into this dns pattern, it maybe a result of the
recent prefetching done by browsers, and you may have a way to resolve the
issue.

~~~
chrisbolt
Is 5000 dns req/s really a considerable amount of resources? We were
previously serving it on old hardware, with negligible (<5%) CPU usage
(tinydns) and bandwidth.

~~~
mryan
If they have a large number of customers all doing 5000 req/s, perhaps they
will need something a bit more powerful.

------
coderdude
I'm surprised the article doesn't actually tell you the meta http-equiv to
disable DNS prefetching. It mentions that it helped out tremendously, though.
Here it is:

<meta http-equiv="x-dns-prefetch-control" content="off" />

or, if you're more into HTML5:

<meta http-equiv="x-dns-prefetch-control" content="off">

~~~
buro9
I prefer HTTP headers instead of meta tags where possible.

An example solution at the load balancer level, assuming the use of Varnish:

In vcl_fetch:

    
    
      if (req.url ~ "\.(htm|html|php)" || req.url ~ "\/(\?.*)?$") {
        set beresp.http.disable-dns-prefetch = "1";
      }
    

In vcl_deliver:

    
    
      if (resp.http.disable-dns-prefetch) {
        remove resp.http.disable-dns-prefetch;
        set resp.http.X-DNS-Prefetch-Control = "off";
      }

~~~
blantonl
do all the browsers that support the meta tag support consuming http headers
for this "switch"?

~~~
metageek
Http-equiv is just a way of specifying an HTTP header in your HTML, so they
really should.

------
joshfraser
Before everyone goes rushing off to disable DNS prefetching, remember that DNS
prefetching is generally a good thing that exists to make websites faster. And
the faster your site is the more pageviews you can expect. Faster sites also
have a lower bounce rate and better pagerank from Google.

~~~
ssp
Does Google take DNS prefetching into account when they measure the speed of a
web site?

~~~
joshfraser
They get their timing data from the Google Toolbar, so yes.

------
Rantenki
This implies that providers are either severely limiting their caches, or
expiring in a shorter than posted TTL. Even though pinkbike looks like it has
thousands of users, one would expect the front pages to be largely identical
for most user sessions, so the ISP dns caches should already have most of
those username.domain.com records cached. Either that or the ISP's DNS servers
are more numerous and distributed, with fewer customers each, or something
like that.

Anybody at an ISP that can fill us in on DNS TTL mangling or cache limiting?

~~~
ars
See my reply here: <http://news.ycombinator.com/item?id=2306788>

He is creating thousands of unique subdomains, so nobody can cache anything.

~~~
drwxrwxrwt
username subdomains are a very popular feature with many websites, especially
older ones. users like it too. there is nothing abusive about it. He cant
scrap several years worth of links and throw everyone at
www.example.com/username

~~~
ars
And? Someone asked why the DNS wasn't caching and I explained.

------
patrickgzill
I am actually rather shocked that people are being charged extra for DNS -
surely the answer is to get any sort of cheap VPS and put DNS on that box?
Then again, why are you depending for all aspects of your site's working at
all, on a service which costs $2/month?

Even a 128MB RAM VPS could comfortably handle a huge number of requests.

~~~
arbitrarywords
Depends on the location of your users. Quicker to have someone host it is
multiple places all over the world (assuming you care about perceived page
load speed)

------
datums
Create an A Record for your highest queried subdomain. Those will be cached
and eventually decrease the number of queries. Depending on the number of
subdomain you can create and remove it on signup and cancellation.

------
ck2
short answer for Firefox

    
    
       about:config 
       network.dns.disablePrefetch             (true)
       network.dns.disablePrefetchFromHTTPS    (true)

~~~
ck2
For those downvoting, did you want the server-side solution?

    
    
        <meta http-equiv="x-dns-prefetch-control" content="off" />

------
andrewcooke
ok i'm confused. why doesn't dns wildcard caching solve this?

~~~
chrisbolt
DNS wildcards are server-side, so there's no way for the client to know that
the response they got is for *.domain.com but not www.domain.com.

~~~
andrewcooke
(I am not an expert on this, so may be wrong, but...) How does that change
anything?

If the client needs to know foo.example.com they ask the nearest server. That
will have the wildcard and will return the correct address. The nearest server
is not going to be the root server for that domain, but something (typically)
at the ISP. So the ISP DNS server will see more load, but not the root. Yet
this article show the root as getting a huge number of hits. The only
explanation I see is that they are not serving wildcard responses when they
should be...

------
gcb
\+ saved 10 billion DNS queries per month.

\- wasted a total of 10sec of each user's time per month.

it's all about trade offs

~~~
Rantenki
No, it's not; the issue is that the domains prefetch REGARDLESS of whether the
links are ever clicked on. The browser is pre-fetching in case the user DOES
click on the link. This really verges on being a browser bug, esp. since in
firefox's case, it fetches both A and AAAA records.

~~~
otterley
The benefit is reaped in the event a link is actually clicked on. That's why
browsers do it.

------
togasystems
As a fellow Mountain biker, I was surprised and glad that I saw pinkbike on
HN.

