
Magic keywords on Google and the consequences of tailoring results - milliams
http://www.gabrielweinberg.com/blog/2012/10/magic-keywords-on-google-and-the-consequences-of-search-tailoring-results.html
======
Matt_Cutts
I'm happy to give more context on this. Some people don't put all their
information needs into a single query. For example, instead of searching for
[iphone wikipedia] to find the iPhone page on Wikipedia, they'll do one search
for [iphone] and then their next search will be [wikipedia].

Google tries to help with those sorts of search sessions. For 0.3% of queries,
if we see a search for a query A and then another search for query B, and
there appear to be good results related to both A and B, then we may surface
those results.

For example, I just did the search [iphone] and then the search [wikipedia].
In addition to the regular results for Wikipedia, Google also surfaces the
page <http://en.wikipedia.org/wiki/IPhone> . A good way to see that Google is
doing this is to look for a phrase like "You recently searched for iphone"
under the newly-surfaced results. Go ahead and try it with the search
[twilight] and then the search [wikipedia] for example.

Between Gabriel's article and the WSJ article, words that are reported to
provide this behavior include iphone, nexus, obama (but not romney, because
there wasn't enough information for this word at the time the data was
generated), tablet, twilight, computer, health, speech, iraq, sports, social
security, and stock.

Just to reiterate, this algorithm affects 0.3% of searches on Google. Most
Hacker News readers are savvy enough to search for [iphone wikipedia] instead
of breaking that search into multiple queries. However, if you don't want
Google to surface additional results that might help with your current query,
Google has a support page telling how to turn off search history
personalization:
[https://support.google.com/accounts/bin/answer.py?hl=en&...](https://support.google.com/accounts/bin/answer.py?hl=en&answer=54048)

As an aside, I wrote a blog post ~4 years ago to preemptively debunk the idea
that Google skews our search results for political reasons:
<http://www.mattcutts.com/blog/google-search-and-politics/> We simply don't do
that.

~~~
donkeydong
What you're saying, though, is that Google is making an inference. DuckDuckGo
does not want to infer anything and I tend to agree. Also, why aren't all of
the inserted links highlighted with the, "you recently search for" line? Why
wouldn't Romney be a magic keyword? I appreciate your explanation but it
doesn't really explain anything.

~~~
nsns
I don't work for Google or have any inside informatgion, but I guess "magic
keywords" become such after a certain (very large) threshold. Obama, being the
president of the United States for four years, has probably crossed that
threshold long ago, but it only became a political issue since the election
process began.

------
patrickaljord
I'm tired of Gabriel Weinber telling me how bad Google is and therefor why I
should use DDG instead. I'm happy with Google, I'm glad that when I search for
"ruby" I get the programming language and that when my brother who works in
jewelry searches for "ruby", he gets the gemstone.

But seriously, tell me why DDG is awesome and unique, stop bashing the
competition. It was fun at the beginning but now it's getting old. Like
basecamp bashing microsoft project at the beginning, it was ok, when was the
last time you read 37signals still writing about how bad MS Project is?
Exactly.

Stop spreading FUD about Google and tell us what's great about DDG. Google
Search is an awesome product that changed the world, it gives great results
and at much greater speed than DDG. Beat them at that and I'll give DDG a go.
Not for any scary reasons.

~~~
epi0Bauqu
Hi, Gabriel Weinberg here. Sorry to offend you, as was of course not my
intention.

I also of course don't think this is FUD at all. As far as I know, this is the
first quantitative study of any kind about the filter bubble. I think it is
hard to dismiss a concept out of hand when it hasn't even been effectively
studied yet.

Also, the filter bubble it is not an effective marketing message because it
requires too much education given it is a complicated subject no one knows
about.

I also think tailoring can be just fine when it is opt-in and you are in
complete control of your data.

As for DuckDuckGo, we've been telling people plenty about new stuff. Most
recently we've been focused on our open-source plugin platform --
DuckDuckHack, <http://duckduckhack.com/> \-- where you can hack the search
engine. And people have been doing it and making cool stuff, e.g.
<https://duckduckgo.com/?q=khan+math> and lots of others:
<https://duckduckgo.com/goodies/>.

~~~
patrickaljord
Thanks for your answer. I'm not offended, I'm just getting some kind of
fatigue reading your anti-google posts. I think insinuating Google care more
about Obama than Romney is bordering on FUD, like all of your anti "bubble"
articles are always bordering on FUD or at least intend to scare people a bit.
You can't deny you're kind of aware of that, aren't you?

It's been more than a year now since you're writing anti-google and anti-
personalized results articles. It's getting old and people love personalized
results. I don't think it should be made opt-in as I think Google (and any
product) should make the best option opted-in by default, and personalized
results is the best option as it offers the best experience for regular users.
Why would they offer a worst experience by default? It doesn't make sense.
Also, most people don't know or care enough about changing options so they'd
just keep the worst experience by default.

Let's take my brother's example again, if he searched for "ruby" and got
results about rails and ruby.rb first before the gemstone because they are
more popular than ruby the gemstone, my brother would just be confused and may
even waste time clicking on the rails link first. So what should google do,
propose a link that would say "opt-in to have search results that actually
make sense to you". Do you see how absurd and bad UX that would be?

People who care about these kinds of things already opted-out, in fact, the
most paranoid hackers I know don't even use DDG, they use Google+Thor and turn
JS/cookies off. It's cool that DDG is adding more features but that's not
anything I care enough to use or I could just write a quick google chrome
script that'd do the same.

~~~
etherealG
why do we have to have an assumption that personalised means doing so in a
negative way like enforcing a political affiliation? perhaps the engine can be
clever enough to realise that some things like localised weather are a
reasonable personalisation, whereas politics are more dangerous and should be
less personalised.

it seems to me that both google and ddg have problems here, ddg in not
personalising at all, and google in personalising things that really shouldn't
be.

~~~
fceccon
> ddg in not personalising at all

I think DDG has some personalization based on your location, for example if I
search "weather" it shows me the weather near me.

------
SoftwareMaven
I'm trying to decide how concerned I should be about Obama being a magic
keyword and Romney not. On the one hand, Obama has been president for nearly
four years; the number of searches for him will have been high and consistent.
On the other hand, Google is _so_ central to information flow today, having it
decide which candidate should get special treatment is disturbing.

I'm mollified somewhat that the Obama magic results that came up when
searching for Romney were from Fox News[1], but I still feel a faint disease
at the whole concept.

I've been using DDG for several months now and like it. It hasn't matched
Google on technical searches, but on general information searches, I prefer
the results to be less biases by me.

1\. I would personally prefer to never see a link to Fox News. However, if
every search for Obama was extolling his virtues and every search for Obama
was extolling _his_ virtue, my unease would be outright disgust and I would be
contacting my Congressman. My unease comes from not knowing _why_ they might
show me something, other than to echo back what I already think, wchich is
noot a very useful set of information. I already know that.

~~~
gojomo
Google sees how people followup queries over time. So they know the relative
frequencies of the four distinct progressions:

    
    
      [Obama], [Iran], [Obama Iran]
      [Obama Iran]
      [Romney], [Iran], [Romney Iran]
      [Romney Iran]
    

The triggering could be as simple as the fact that {[Obama], [Iran], [Obama
Iran]} happens more often than {[Romney], [Iran], [Romney Iran]}.

And that difference could be because people searching one path are slightly
more likely to keep refining simple queries ("try, try again"), versus other
people more likely to combine-up-front ("measure twice, cut once").

------
kevingadd
Most of the stuff discussed in this post didn't bug me, but then I got to this
part:

 _As the Wall Street Journal confirmed in its own study, Google has been
significantly altering its search results to highlight Obama-related results,
but not Romney-related results (more on that later).

These Obama-related results are being inserted because obama is a magic
keyword on Google. A magic keyword is a search that can transform the Google
results of later searches._

Incredibly shady. Changing the results of later searches based on things I
searched for in the past is the _complete opposite_ of what I want a search
engine to do.

~~~
dsr_
I wonder if the effect times out? If so, it could be reasonably construed as
an attempt to present more relevant results across a session.

If, on the other hand, Google accumulates history until you sink into a morass
of similar results, it's clearly harmful.

~~~
jmillikin
In the original WSJ article (yc:
<http://news.ycombinator.com/item?id=4741394>), it's confirmed that this
effect has a limited duration.

    
    
      > A Google spokesman said: "We aim to get users the best
      > answers as fast as possible" using techniques such as
      > examining "related searches." He said the goal for the
      > feature is to provide better results in a situation
      > where, for instance, a person who searches for "Harry
      > Potter," and then for "Amazon," actually wants "Harry
      > Potter" results from Amazon.com Inc. He said that the
      > technique saves Google users time and provides better
      > answers, but affects only about 0.3% of the searches the
      > company conducts.

~~~
oddjob
That's not confirmation, that's a public relations response. _Confirmation_ of
limited duration would be actually running the experiment until the effect
actually disappears.

------
davemel37
I ran the search and I am not concerned at all... Google shows poll results
with bright red and blue colors drawing the eye... News stories show up first,
and pictures of Romney, and his website outrank these related results.

The "you recently searched for obama" results appeared below the fold and the
story was quite frankly very unlikely to be noticed unless it was something I
was looking for specifically, which is probably google's algorithm's intent.

I tried running the search in battleground states and the same results showed
up and I tried a bunch of searches like "Romneys plan" "why vote for romney"
"who should I vote for" "vote for romney" and none of them showed obama
results...

P.S. I am a libertarian who voted for Romney and I approve this message.

------
BrianPetro
"65% of people said personalized search was a 'bad thing'"

So does this guy (
[http://www.ted.com/talks/eli_pariser_beware_online_filter_bu...](http://www.ted.com/talks/eli_pariser_beware_online_filter_bubbles.html)
) on TED and will easily convince you if you aren't already.

~~~
angryasian
if people really wanted to find the other side of the story its available to
them. I absolutely love personalized search. When I search for something I
want local results, I want results that are most relevant to me. Its been
proven that people would only go to sites that support their ideas and beliefs
already. So showing unbiased results make no difference.

~~~
skinnymuch
So if things are already messed up, we should just throw our hands up, pander,
and go along with it?

~~~
angryasian
whats messed up ? I stated I like personalized search because 9/10 times I'm
googling something its related and I'd want to be personalized , either by
local or a topic i've searched before. Not personalizing results will probably
make search worse and people less likely to search in the first place.

------
eli_awry
Assigning a moral value to Google's algorithms here, or to personalized search
in general, doesn't make sense. I _like_ personalized search - when I change
main operating systems, as I do every few months, my information for debugging
is always for the right os. If I look up hashmaps, it usually gives me results
for the programming language I'm using that day. In the middle of a study
session for linear algebra, I can just search for inverse and get a useful
wikipedia page. If I want results that are not personalized like this for some
reason - more balanced coverage, I actually want to know about pythons,
whatever - then hopefully I have the discipline and ability to go to incognito
or another search engine. Maybe I don't know about them, in which case if I'm
politically engaged I will find out from a fearmonger. But the reality is that
there are two products, each ideal for different people and use cases, and
neither of them is evil. It has always been difficult to get 'unbiased'
information, whatever that is. With Google, I have a chance to see beyond my
local papers and the people I know in one way, and with DDG it's another.

------
dgunn
"We chose to use these keywords (abortion, gun control and obama) because they
are both a) searches where many people want unbiased results..."

I suspect that someone searching for these keywords are looking for
reinforcement of their predetermined beliefs. Which is exactly what
personalized search would do.

[edit] I've been downvoted so I feel I should clarify. I'm saying that these
keywords are "magic" for a reason. This test should be performed using
keywords people _actually_ want unbiased results for. A good example would be
a good javascript library for making graphs. (something I searched for
yesterday and wanted to see an unbiased comparison). These keywords return
biased results because they are highly polarized topics and you probably
situate yourself on one of those poles - not in the middle. If google knows
this about you, good for them and you because you found what you were looking
for.

~~~
skinnymuch
Filter bubbles are troubling for exactly the reason you don't seem to have a
problem with.

It's not particularly good for the world to have Obama searches be filled with
either Fox news results or MSNBC (pro-Obama/Dem) results.

------
kulkarnic
I wish Gabriel had inserted a map of where the screenshots were taken. My
hunch, which could be wrong, is that magic keywords are geographic too. In a
state like NH which leans strongly, maybe the algorithm automatically
identified Obama as a magic word. (This hunch is based based simply on ddg
being not-quite-mainstream, so maybe its users are more democrat than the
country?)

I think it'll be interesting to see the Google response to this. I think the
most important question is whether identifying magic keywords happens
automatically, or is programmer input involved?

------
the_gipsy
DuckDuckGo is at it again, this time smearing politics into tailored search.

------
wild91
What if I turn off Google Web history?Doing this should resolve tailoring
result,or not?

------
Teapot
Google uses the word 'personalization' as a euphemism for 'censorship'.

~~~
goloxc
No Free Lunch!

------
capo
Just so that everyone is on the same page on what is being discussed here:
this is a “study” by a competitor who can't implement personalised search and
thus is spreading FUD to spin that as a differentiating advantage. It might be
a good marketing strategy as he managed to get coverage by a couple of outlets
during the elections frenzy in the US, but this "filter bubble" buzz-phrase is
nonsense. Don't fear personalized results they are just another way of sussing
relevance, and as mentioned in that post it concerns a tiny number of queries
and has minimal effect on the results page.

~~~
AJ007
"We also saw in Google chrome that magic keyword transformations sometimes
jumped incognito sessions, meaning that if you started a new incognito mode,
got a transformation, then shut it down and started a new incognito mode you
could sometimes see the same transformation again (without searching for the
same magic keyword again). This weird behavior was not reproducible in
Firefox's private browsing mode."

If that is FUD, that is some pretty damn creepy FUD.

~~~
magicalist
Yeah, that jumped out at me as well, and actually seems somewhat irresponsible
without more data (like a log of actual http traffic).

AFAIK (when I last checked), Chrome keeps no data between incognito sessions.
If data is being kept, it should be extremely easy to spot and a bug should be
filed. "This weird thing happened" isn't really a sufficient response to that
theory.

If there isn't data being kept and transmitted, it means either the testers
left open an incognito tab or something without noticing (easier to do than
with private browsing, since private browsing closes all non-private browsing
tabs until private browsing is over), or that google is tweaking results based
on search history saved server side by ip address (and maybe other id-able
browser characteristics), but _only_ if the browser identifies itself as
Chrome, not e.g. Firefox. There doesn't seem to be any real advantage there
(and you'll likely just end up polluting anything but the coarsest of
clustering), but that's just speculation.

I'm not able to reproduce the behavior in Chrome (though I'm having trouble
reproducing even simple magic keyword behavior in incognito mode in the first
place), so more data would be appreciated.

------
Evbn
Maybe Google is not the problem if a voter's idea of political engagement is
typing in "one word" and voting for whatever pops up?

