
Care about search privacy? Duck Duck Go no longer logs IPs. - epi0Bauqu
http://www.gabrielweinberg.com/blog/2010/03/care-about-search-privacy-use-duck-duck-go.html
======
devinj
It seems sort of like you're destroying the possibility for growth with this,
though. From what I understand of Google's work, they really have improved
tremendously from analyzing data (to find how searches go wrong), and from
using the data in new features (such as with spell checking, adding new terms
into the query, personalizing searches, etc.).

And I mean, I value privacy. I do. But, I value the quality of the search
results even more than whatever privacy is lost from search engines logging, I
guess. Unless there is some path to improvement, all I can ever see myself
using Duck Duck Go for is porn, tips on murdering my spouses, and information
on illegal drugs. Of course, other people would use it for much more-- but for
me, there something truly wonderful about being able to search for "nose" and
have the first result be a unit test runner I want to use, not a piece of
anatomy I already know everything I need to about. Sure, it's stupid-- I
should search for "nose tests", but I sometimes make mistakes, and here Google
doesn't even need to say so and make me try again. It finds the target anyway,
without problems (and the less-likely Wikipedia article is right below, as
result #2). Without Google logging what I searched for and building a profile,
this wouldn't be possible.

I don't mean to be fanboyish and loving of Google (they do scare me
sometimes). If they added some sort of feature that said, "don't record
anything I do right now with these searches", I would be very happy. But,
similarly, I would also be happy if Duck Duck Go at least had an option to
opt- _in_ to logging and monitoring and so on. On my end, I would consider my
relatively minimal, and explicitly given, loss of privacy to be worth it to
help make a search engine I am using better (assuming, of course, that I'm
using it-- and I'm not opposed to trying something new to me).

~~~
lena
In my opinion Google results used to be great before they did all the
personalizing, and now they are often really bad. I get infuriated sometimes
how hard it is to search for a specific term. I have to use all kinds of + and
- signs in my search terms to get anything near a meaningful result and it is
not straightforward at all to simply go to an English version of a Google site
because Google insists that I must be Dutch and therefore only want localized
information.

Of course they did not get worse because of the personalizing (though the
location-based stuff absolutely has to go away - we have browser language
strings for that, don't try to guess what I want if I have a way to tell you
what I want), but I do think they did pretty well before they did all that, so
I see no reason why DuckDuckGo can't as well and wish them the best.

~~~
lionhearted
> it is not straightforward at all to simply go to an English version of a
> Google site because Google insists that I must be Dutch and therefore only
> want localized information

This kills me when traveling. There's ways around it, but it's a bit of a
hassle. Really one of my only hassles with Google, though I understand it
probably serves 99%+ of searches better.

~~~
evannyx
Try <http://www.google.com/ncr>

------
ErrantX
This is definitely a killer niche to fill given the current privacy conscious
climate [around here]. Kudos.

~~~
ryandvm
"Privacy conscious climate"? Are we using the same Internet? (The one with
Twitter, Facebook, Foursquare, etc...)

~~~
ErrantX
By privacy conscious I mean users - specifically us techy users (where there
is a lot of privacy debate currently going on)

~~~
webwright
It's certainly a niche, but I'm not sure how many techy users really care
about privacy. We all have Gmail accounts, right? Which store emailed
passwords, password reset links, private conversations, account balances, and
all sorts of other stuff?

Another data point... Our userbase at RescueTime is comprised almost entirely
geeks. We measure and store second-by-second attention data on their computer.
We make all sorts of privacy features available (ability to delete bits of
data, track only whitelisted stuff, etc., etc.). The VAST majority of our
users never touch any of these features.

And we're growing like gangbusters.

I think people who make the bet that people in the future are going to be
increasingly privacy-conscious are making a pretty bad bet... But I do imagine
that it will continue to be a niche (though likely a shrinking one).

~~~
jacquesm
I think you are right but in the short term only. I think long term there will
be some huge flap involving online privacy and after that people will wake up
to what the consequences can be.

~~~
webwright
Really? What can the consequences be? I'm not being flip-- I think it's easy
to say that but its harder to pin down dangerous scenarios.

The increasing fact is that the more Google knows about you, the better they
can serve up search results (and ads, of course). I think most users would NOT
opt-out given the chance to do so (if it meant harming their search quality).

Related quote from Matt Cutts' blog:

"It’s a long article, but an example useful fact is that if X is the number of
people who visit the Ad Preferences page and opt out, 10X people don’t opt out
and 4X people actually edit their categories to improve the targeting
relevance of the ads they see. Let me say that again: four times as many
people change their settings to make their ads _more_ relevant than opt out of
interest-based targeting. I think the Ad Preferences page is a good example
where users get more transparency and control regarding their privacy."

~~~
jacquesm
Let me give you just two possible scenarios, there are plenty more:

\- An insurance company somehow gets their hands on search data and finds a
way to associate that with their customers, figures out who is probably about
to apply for coverage for a bunch of expensive treatments and cancels their
insurance before they can make the claim for some trumped up reason.

\- A political party gets their hands on this information and datamines it to
the hilt in order to figure out which talking points to trot out on what TV
stations in order to swing a maximum block of voters to them.

I'm sure there will be people that will say both of these are unrealistic, but
I think that given the amount of data out there it is only a matter of time
before someone will find a way to abuse it in a way that we will not be able
to ignore.

~~~
froo
>An insurance company somehow gets their hands on search data and finds a way
to associate that with their customers

You mean like the AOL search data leak? I never read anything about insurance
companies misappropriating the data in this manner. So I'd say its more of a
long shot.

Same with political parties... again, never read anything like that.

Not to say that it hasn't happened, I'd be curious if anyone has any
information to show if it has?

------
rgrieselhuber
I noticed you're running your own servers. What kind of setup do you have? How
many machines? Are the machines literally sitting in your garage or are you
colo'ing somewhere?

Not asking because of privacy, just curious about the technology. If you've
answered these questions already somewhere, sorry!

~~~
BobbyH
He wrote about his site's architecture here:
[http://www.gabrielweinberg.com/blog/2009/03/duck-duck-go-
arc...](http://www.gabrielweinberg.com/blog/2009/03/duck-duck-go-
architecture.html)

He also answered some questions on HN here:
<http://news.ycombinator.com/item?id=525048>

------
whatismyip2
He does log the browser string. The EFF's panopticlick project shows that this
is a way to identify a user surprisingly uniquely.

~~~
epi0Bauqu
Wow, really? Any references? I'd consider ditching them. They're not doing
much for me anyway.

~~~
bensummers
<https://panopticlick.eff.org/>

~~~
epi0Bauqu
Right, that's tracking everything the browser outputs, not just the user
agent, which is maybe 1% of the full string.

~~~
whatismyip2
I did an experiment and turned off javascript and redid the test... it still
is unique enough to track a browsing session. Give it a try.

I would consider using Duck Duck Go if nothing was logged. Right now I use
another service precisely because they don't log anything.

~~~
epi0Bauqu
I just turned user agents off.

~~~
whatismyip2
Wow, now that's what I call customer service :-) Thanks for listening... I'll
give it a try.

------
alecco
How is the legal situation in US? Aren't web services forced to keep records?

~~~
qeorge
No, but there are rules if you do. (e.g., have to share with FBI when asked,
have to delete after X months).

Most search companies want to identify you. The problem is keeping them from
having too many records.

~~~
axod
>> "Most search companies want to identify you."

I'm not convinced they want to do it for any malicious/profit based motive.

They want to identify me to give me a better user experience, sure.

Being able to identify users searching on a search engine doesn't suddenly
mean it's easier to monetize. Google would still make billions from the
adverts next to search results regardless of how identifiable users are. It's
not related to their bottom line.

I guess I just don't buy the conspiracy theories on it.

~~~
mapleoin
Storing that much data about users isn't harmless just because the company
doesn't do anything bad with it.

Even if that company would forever be _good_ , they could still have their
servers infiltrated into or stolen, or some government could decide that they
want the data over night.

~~~
axod
If you're looking at risk though:

    
    
      * Computer could have been compromised
      * Wifi/local network could have been compromised
      * Your ISP could have been compromised
      * Some of the peers on the internet between your ISP and google "
      * The datacenter where Googles servers are
    

And that's before the data gets to Google.

I'd guess that the most common privacy theft is at the first 2 steps - local
computer / wifi network.

Any one of those could happen, someone could be logging all your searches +
all internet traffic in some of those cases.

Personally though, "What I've searched for" is in the 'meh don't care if
anyone knows this' bucket for me.

~~~
y0ghur7_xxx
just because your data can be stolen in other ways does not mean you should do
nothing about it. you could address all those issues.

~~~
axod
Right, but knowing the risk associated with things is important... For example
I don't care much about terrorism, I care about making sure my kids look both
ways when crossing the road.

Similarly, I care more about individual users PCs getting compromised than
Google getting hacked.

------
acg
There's still an issue of trust, why should I trust Duck Duck Go any more than
a company that has a reputation to lose?

Quis custodiet ipsos custodes?

~~~
Goladus
That's a question you have to answer yourself. Right now, Duck Duck Go is the
only search service I've ever heard of even trying to make that promise.

As an aside, there are a couple issues with companies and reputation. First is
that a company might risk its reputation if the rewards are worth it. Second
is that a company might care about its reputation but simply fail to hold true
to it. Finally, a company might care about its reputation some of its
employees might not.

~~~
sounddust
[http://arstechnica.com/old/content/2007/07/ask-com-to-
offer-...](http://arstechnica.com/old/content/2007/07/ask-com-to-offer-
anonymous-search-with-askeraser.ars)

------
MikeCapone
If you still want to use Google but want privacy, try:

<https://ssl.scroogle.org/>

What they do is explained here: <http://www.scroogle.org/> and here
<https://ssl.scroogle.org/sslnote.html>

------
JustRick
What about your upstream provider? Do you know what level of data they retain
about your traffic and visitors/sessions?

~~~
epi0Bauqu
Doubtful any. It's FIOS business, and I'm directly connected via to their
fiber backbone.

------
ximeng
epi0Bauqu you may want to add DDG to Firefox's list of search engines (drop
down search engine box, manage search engines, get search engines):

<https://addons.mozilla.org/en-GB/firefox/search-engines/>

Didn't see it there.

~~~
epi0Bauqu
Hmm..we do have a Firefox add-on and I am working on another one. I'm not sure
why it doesn't show up there though. Thx.

------
fexl
I have duckduckgo set as my default search engine in Firefox. Does anyone know
how I can make it use SSL? In other words, make it use
<https://duckduckgo.com> instead of <http://duckduckgo.com>.

~~~
est
In chrome you just add an 's' in the URL in your default search engine
settings.

~~~
fexl
Aha, I did this:

    
    
      find ~/.mozilla -name "*duck*"
    

Then I edited duck-duck-go.xml, changing http to https, and restarted Firefox.

------
aarongough
Great to hear! I have been considering switching to DDG to give it a long-term
try for a while now and this pushed me over the edge! DDG is now my default
search site in FireFox!

Great work epi0Bauqu!

------
Xurinos
I use ixquick, and they have SSL, too: <https://ixquick.com/>

I am curious which of these will grant more privacy and better searches.

------
moultano
Unfortunately, if your search engine becomes important enough to spam, you
won't be able to use logs as a signal . . .

Best of luck though. Seems like a good publicity strategy.

------
mike-cardwell
The search results page retrieves some content from s3.amazonaws.com. The http
referer in those requests contains the search term. Thus, even though
duckduckgo.com doesn't know you performed a search, amazonaws.com knows your
IP address, what you searched for, and when you searched for it. They
_probably_ have a log of all that information too.

~~~
epi0Bauqu
I'm in the process of changing these links to https.

------
orblivion
So the one thing that never seems quite right about this: how/why do we trust
that the guy doesn't log IP addresses? Let's put it this way: If someone
promises not to do anything bad with the IP addresses they store, and we don't
trust them about _that_ , why would we trust someone when they say they don't
store them at all?

~~~
Goladus
Flip the question around for a moment. Why aren't we all super paranoid about
Google? Because we more or less believe that the way they use the information
they collect isn't damaging to us. It doesn't mean we trust them completely,
100%, in every way, shape or form. It doesn't mean we don't use google's
service with some degree of reservation that using their service is a bad idea
even if they are telling the truth.

If I didn't have a basic level of trust with google I wouldn't use the
service. I can apply that same basic level of trust with duckduckgo.

Once you apply that unbiased trust, you have two services: one that uses a
variety of technology to protect the information they collect and one that
doesn't collect information at all and takes steps to prevent others from
doing the same.

Google might get bought out. Google might be subject to a sopoena. A google
employee might steal something. Google might make a mistake. Any one of those
things might prevent Google from fulfilling the basic promise of "we won't
misuse your data." They won't ever have to lie or do anything evil. They might
just make a mistake, and all your data will be there, vulnerable.

~~~
orblivion
Which did happen once with AOL searches. I see your point.

------
jacquesm
That's great news. Pretty ballsy move too, I'd like to see google match you on
that one.

The one little issue is that IP traffic passes through a whole pile of routers
before reaching your target, so if you _really_ care about your privacy that
much then you should probably use https.

~~~
orblivion
Ballsy, I don't know. I'd call it very smart. Maybe I don't know the extent of
the benefits of IP tracking, but being relatively unknown, this is a great way
to gain market share as a niche.

> you should probably use https

Not sure if this is what you're implying, but <https://duckduckgo.com> is
mentioned in the article.

------
mkyc
Good move, you now have one more user to not-track. I'll be trying DuckDuckGo
out as my default search. I'll give you feedback if it doesn't work out. I
hope others will give you a try by setting you as the default, too - you can
always switch back. (Those that do, change http to https.)

------
adrianwaj
I prefer Google. The full link is in green, it's justified left, the results
are wider and don't take as much vertical room. There's a cache too. What does
DDG do better than Google? Does DDG have the crawling power of Google or does
it pull in Google's results?

~~~
Goladus
_What does DDG do better than Google?_

SSL connection. Nothing logged. When I don't care about results
personalization, and if I get results I am looking for, DDG has the superior
service.

Today, if an employee working for Big Evil uses google to do a search for
"Software Developer open positions near Cambridge, MA", that search will
likely be logged by websense and the internet firewall. That information might
be found and used against you by a political enemy (when you work for Big
Evil, political enemies emerge from the woodwork).

A search made over an ssl connection won't log the details.

~~~
adrianwaj
Will the SSL hide a post/get? The SERP still have the terms in the
querystring. You can get SSL on other Google apps. Maybe not on search.

~~~
paulbaumgart
Yes, it will hide them. TLS runs at a protocol layer below HTTP:
<http://en.wikipedia.org/wiki/Transport_Layer_Security>

------
greenlblue
duck duck go just became my #1 search engine.

------
rsheridan6
Nice. A few searches turn up relevant results. But I think it should return
more than 5 results.

~~~
ugh
Try scrolling :)

------
krainboltgreene
I don't need to care if people retain my IP. I'm on a netbook and mobile 99%
of the time.

~~~
epi0Bauqu
Even so, your sessions would be obvious in the log, i.e. one could tie
multiple searches together. But to your broader point, some people care about
privacy, some don't.

------
hcho
Am I being cynical with noticing that this comes right after your brand
awareness post?

~~~
epi0Bauqu
Yup. I meant to write that post two weeks ago when I did the poll, but got too
busy.

This came about from comments on reddit. I gave it a lot of thought over the
past few days and decided to pull the trigger.

~~~
hcho
It was a tounge in cheek comment really. This move will definitely give you
some leverage with a very vocal crowd who has concerns over privacy. I really
would like to see more search engines around. I am just tired of big G's de
facto monopoly status. So well done and keep it up :)

------
javery
What was the original reason to store IPs?

~~~
epi0Bauqu
Analytics. Google et al. also use it for personalization, though I never did
that.

~~~
javery
So did you ditch any sort of tracking? It doesn't look like you added cookies
or anything. (very cool if so)

~~~
epi0Bauqu
Yeah, I ditched it.

------
bradgoat
Loving this search engine! Even works great on my Samsung Jet mobile phone
browser (webkit based).

------
freetard
Just turn off cookies and use a proxy, Google results are still way way
superior.

