
DuckDuckGo Searches Are Not Anonymous - mike-cardwell
https://secure.grepular.com/DuckDuckGo_Searches_Are_Not_Anonymous
======
epi0Bauqu
In the settings, <http://duckduckgo.com/settings.html>, you can turn on POST
requests as well as disable favicons and 0-click. Switching to POST alone
should fix this issue for you. In particular the Referer heading then becomes:

Referer: <https://duckduckgo.com/>

~~~
jimboyoungblood
how about you set up a separate domain (e.g., supersecretduck.com) where all
the options you mention above default to their paranoid settings?

~~~
MikeCapone
If that separate domain can be used in the Chrome URL bar, count me in.
Thought it would be even better if it could be done using the main URL in the
Chrome URL bar somehow...

~~~
baddox
If the domain exists it can be used in the Chrome URL bar. Any search engine
that puts its queries in the URL can be added manually to Chrome.

~~~
fhars
The whole point of using POST is to remove the queries from the URL, though.

~~~
Pistos2
In Opera, when setting up search engines [which can be used directly in the
address bar], you have the option of using POST instead of GET.

------
jacquesm
Good catch, really. Trivial to fix fortunately. Not that I care too much about
super secure secret searches, but if they're advertised as such they should
be. And even then, you probably should assume they're not. (secret).

~~~
imp
I can't figure out what the trivial fix is. Would proxying all S3 requests
through his server fix the issue, or do the headers get passed through to
Amazon anyway?

~~~
nrj
One fix is to not directly send the users to the result page. Instead link to
a redirect script on the ddg servers i.e,
duckduckgo.com/goto.php?link=<http://search-result.com/> and then have
goto.php remove the REFERER from the request headers.

~~~
jacquesm
Not sure if that will work in all browsers, iirc a 301 or a 302 can still pass
those headers on. The only trick I know of that will not do that is by using a
'meta refresh' with a time set to '0', but that has bad implications for the
working of the 'back' button.

~~~
nrj
I haven't tested this out, but I don't see why something like this wouldn't
work...

<?php

    
    
       header("Location: the-result.com");
    
       header("Referer: ");
    

?>

~~~
jacquesm
Referer is a header the browser sends, Location is a header the server sends.
Also, the Location header needs either a relative url on the local machine or
a fully qualified one. In this case it would have needed a fully qualified
one.

So maybe you should have tested it ;) ?

------
mey
If super secret squirrel searches are your need, you may wish to look at
things like, disabling Referrer, Tor, Disabling Flash completely (as it
represents it's own version of a cookie), disabling cookies, regularly DBANing
your system, and even then realizing that you are still screwed according to
EFF's research in their Panopticlick project (<https://panopticlick.eff.org/>)

~~~
danieldon
True. torbutton does a lot of that, however, so if you enable it and
completely disable javascript you can get your browser uniqueness down to
about 1 in 1500

------
eli
The heck are you talking about?

Sure, Amazon S3 could be spying on your searches just like any web host
(especially a cloud host) could spy on connections to its customers' sites. I
don't think this is very likely, however.

Nothing shown in this blog post indicates that DDG's S3 account is logging IPs
or that there's anything at all wrong with the privacy policy.

~~~
mike-cardwell
You misunderstand. The images are being served from Amazons web frontend to
S3. They probably log the requests, not for "spying" reasons, but simply
because that's what people tend to do when serving websites; log the website
requests that are made against it. Because they do that, a government could
contact Amazon and ask them for the data.

Knowing that, DDG may as well log the IP+Search themselves, as it makes no
difference. The data is already logged and retrievable by contacting Amazon,
therefore what is the point in DDG not logging it anyway?

------
dzohrob
Isn't this sent by the user's browser, and not by DDG? It's a client-side
configuration issue -- turn off referrers in your browser. Your browser would
send the same header if you clicked through a DDG search result.

~~~
jacquesm
It is a duckduckgo issues because they should not be loading third party
graphics. The referrer will be on in almost all cases. People just look at the
'lock' item and will assume they're safe.

Whatever happened to that 'mixed content' security warning, I thought that was
pretty effective against stuff like this?

~~~
blasdel
Because complaining about "mixed content" was completely pointless in the
general case -- you could mix content across multiple https sites, but the
certificates were never correlated, and the second site would still get all
the headers just the same.

It's extraordinarily user-hostile, and would just add to the pile of pointless
wankery that keeps people from using https (see also: shitting all over self-
signed certs when in reality the CAs don't do shit for their rent and identity
is useless anyway).

The actual solution is to never send Referer headers for cross-site requests
from an HTTPS page.

~~~
jacquesm
> The actual solution is to never send Referer headers for cross-site requests
> from an HTTPS page.

That should be on someone's todo list at the major browser vendors. You're
right, there really is no point in sending that header along, and sending it
can cause all kinds of trouble.

------
JacobAldridge
"First they ignore you, then laugh at you. Then they fight you. Then you win."
(Gandhi via Robbie Williams)

Rightly identified and well addressed by epi0Bauqu. And also worth noting that
this sort of constructive criticism is a great sign of positive market
traction. I'm bumping into DDG more often on the web, which is excellent.

------
nl
POSTs and HTTPs are good, but here's a few other ideas:

1) Embed the images in iframes, which are then embedded in the page. The
iframes will swallow the referrer, so provided they are hosted somewhere where
logs are discarded then it should be fine (I'd want to cross-browser test this
before relying on it though).

2) If the browser supports data: URIs, then embed the image in the page.
Obviously this might have some costs, but you could do it for HTTPS only
perhaps?

3) Request the images in Base64 encoded form (or binary strings) via
XMLHttpRequest after the page has loaded. You can overwrite the Referrer
header in XMLHttpRequest

4) Preload the images BEFORE the search is done (ie, on the search page). With
appropriate headers Amazon won't see a request (this won't work from the
browser bar, though. I could imagine some ways around that, but I'm not sure
they are worth it)

~~~
est
> 1) Embed the images in iframes, which are then embedded in the page. The
> iframes will swallow the referrer, so provided they are hosted somewhere
> where logs are discarded then it should be fine (I'd want to cross-browser
> test this before relying on it though).

https will swallow referer automatically.

~~~
nl
Which is why I said "..HTTPS is a good idea".

There are valid reasons why serving everything under HTTPS isn't always a good
idea. The obvious one is CPU cost, but cache performance can also be affected.
See, for example:
[http://blogs.msdn.com/ieinternals/archive/2010/04/21/Interne...](http://blogs.msdn.com/ieinternals/archive/2010/04/21/Internet-
Explorer-May-Bypass-Cache-for-Cross-Domain-HTTPS-Content.aspx)

<http://blog.pluron.com/2008/07/why-you-should.html>

(I'm not saying that https isn't the best option. I'm just pointing out other
options that can work with plain http.)

------
grhino
Almost all web servers keep a log of HTTP requests if only to weed out
troublesome pests.

The worry is not that the information is sent. The headers, IP address, and
search query is always sent to any search engine regardless of their privacy
policy or whether it's sent as POST or GET, so the worry is not that
information is being sent, but rather that the query string in GET requests is
most likely kept in a log file at least for a few days.

If Duck Duck Go was upfront about how long this HTTP request log was kept,
would that make the default search with GET requests acceptable? I think
having search queries sent by default as POST would be irksome for a default
setting.

------
codexon
This isn't DuckDuckGo's fault.

People serious about privacy will be using a proxy with flash/javascript
disabled and headers scrubbed.

------
jasonlotito
The complaint is that the search terms could potentially be logged.

"They certainly could log that information if they wanted to."

This is a problem with ANY site and ANY system that I know of.

~~~
mike-cardwell
They basically say:

"We don't log your IP address with the search term"

As it stands, they should append this to the end:

"but the architecture of our site lets Amazon log it."

~~~
omd
>As it stands, they should append this to the end:

>"but the architecture of our site lets Amazon log it."

This is the best suggestion yet. Of course there are numerous hacks possible
to make it more secure, but each one at the cost of user experience. DDG has
so many things going for it, I don't think for the majority of users security
is at the top of their list.

~~~
mike-cardwell
Do a http redirect to a url which has an encrypted version of the search term
in the url parameters before displaying the search results. That fixes the
problem without hitting the user experience.

------
jules
Well doesn't duckduckgo use other search engine's results and combines them
with its own? Or does that not involve sending the search terms to the other
search engines?

~~~
jacquesm
Yes, but then it is duckduckgo.com doing the query. Here they're passing on
the query in the referrer string and the users original browser makes the non-
secure request to amazon.

That sort of gives it all away.

So if duckduckgo would just send back the answers to the query and not use a
page that requests resources from third parties they'd be fine.

~~~
dzohrob
Except if the same user clicked through to a result, the server of the result
page would receive the same referrer header w/ the DuckDuckGo query in it.

~~~
jacquesm
Not if he makes the 'POST' option the default for secure searches.

------
jheriko
Once again, bad defaults == class A bug.

Well exposed.

