
Google indexing DDG search results - vincent_s
https://www.google.com/search?q=site%3Aduckduckgo.com+intitle%3A%22at+duckduckgo%22
======
crazygringo
And?

Their robots.txt allows it [1] -- it only disallows URL's with a question mark
in them (i.e. GET parameters) and none of the indexed pages have that:

    
    
      Disallow: /*?
    

And Google is indexing pages like:

    
    
      https://duckduckgo.com/Boeing
    

So not sure what the point of this submission is?

[1] [https://duckduckgo.com/robots.txt](https://duckduckgo.com/robots.txt)

~~~
saalweachter
Yeah, the
[https://en.m.wikipedia.org/wiki/Robots_exclusion_standard](https://en.m.wikipedia.org/wiki/Robots_exclusion_standard)
dates from a more civilized time, when URLs were more, erm, _uniformly_
structured.

------
wmf
Google has a policy of not indexing search results so these will probably be
removed although technically there's nothing wrong here.

------
zamalek
It would probably be more conspicuous if they didn't.

------
theboulevardier
If you look at the cached Google page, you will see that Google is only
indexing the knowledge panel, not the results themselves. E.g
[https://webcache.googleusercontent.com/search?q=cache:t8Z5T9...](https://webcache.googleusercontent.com/search?q=cache:t8Z5T9lBLDQJ:https://duckduckgo.com/Database+&cd=10&hl=en&ct=clnk&gl=uk)

~~~
lgats
Looks like the query results come from a url that includes a ? and therefore
they are blocked by the robots.txt "/*?" rule.
[https://duckduckgo.com/d.js?q=%C5%8Ctsuchi&t=A&l=wt-
wt&s=0&c...](https://duckduckgo.com/d.js?q=%C5%8Ctsuchi&t=A&l=wt-
wt&s=0&ct=US&ss_mkt=us&vqd=3-98318517210371365151044395550013262794-162317560025750407458371726853926388732&p_ent=&ex=-1&sp=0&v7exp=a)

------
cocktailpeanuts
Have you ever thought, maybe that's how DDG intended it? small time search
engines like DDG have no luxury to block any potential routes of people
discovering them, especially through Google. I would even bet they are doing
"SEO" to optimize their search results for Google.

------
InsomniacL
Is their robots.txt valid?
[https://duckduckgo.com/robots.txt](https://duckduckgo.com/robots.txt) # No
search result pages Disallow: /*?

~~~
rbinv
It is valid, and so are Google's results as the URLs don't contain a question
mark.

------
utopcell
It almost looks staged.

------
raslah
It's weird that there some subtle defensiveness here. To heck with robots.txt
and what not, when a site touts it's privacy, even this, however irrelevant,
is slightly eyebrow-raising.

~~~
kchamplewski
How does this in any way reflect on either the privacy of DuckDuckGo (I don't
think they ever claimed or even vaguely implied that if I search for, say,
Banana, no one else will be able to see the results page for Banana, as that
would be absurd), or Google, who are simply indexing publicly accessible
pages, which is what they've always done since coming into existence.

What violation of privacy or hint thereof is happening here?

~~~
raslah
None. Thus my question of why the Curt responses and passive aggression. DDG
has positioned itself as the anti Google and lo and behold Google still finds
out what's being searched there, though not necessarily who searched it. If
you're a bit cynical, it's vaguely interesting.

