

Study Suggests Google Harms Consumers by Skewing Search Results - caminante
http://on.wsj.com/1SZhSpn

======
dpflan
From yesterday:
[https://news.ycombinator.com/item?id=9798256](https://news.ycombinator.com/item?id=9798256)

No comments there, maybe some will arise here.

~~~
moey
Why does it matter? I'm tired of seeing people link other discussions. If it's
a _live active_ discussion, thats one thing, but a dead discussion from
yesterday?....

~~~
jemka
The comments of a submission are valuable to those other than the
participants. To readers that don't or won't comment, the concept of 'live
active discussion' is moot. In fact, I could argue that it's most beneficial
for a reader to wait until the comment activities die down.

Either way the link to another similar thread is an odd thing to be tired of.

------
downandout
The key statement in this article, that renders this report largely useless,
is of course contained at the very bottom of it:

 _" The authors conceded that there were differences between their click
surveys and actual search results, and that they didn’t have access to
internal Google data to verify their results."_

In other words, their conclusions are meaningless because they simply don't
have access to to the data to determine how Google would rank search results
in the fictional scenario that they were attempting to study. They essentially
did A/b testing, showing some users fictional, Yelp-heavy search results
injected into a fake Google results page, and compared those CTR's to those of
an actual Google results page. While the fake pages achieved a higher CTR,
that would probably be the case for many custom-crafted results pages curated
by humans.

This was a "study" that was paid for by Yelp, in their quest to force Google
to feature their results more prominently. I guess when you build your entire
business on quasi-extrotion, heavy handed tactics come naturally. But this one
falls completely flat for me.

~~~
eridius
Maybe I'm reading it differently than you, but I don't see that statement as
being particularly meaningful. All that says is that they didn't have access
to Google's actual internal data on clickthrough rates. Which is not
surprising.

It turns out what they did was use a site called UsabilityHub (which is
intended for testing website designs) to show the study participants a
screenshot of search results. One screenshot showed Google's results, while
another shows search results that were purportedly sorted purely by relevance.
I'm assuming users simply reported whether they would have clicked on any
results on the page, but the article doesn't go into that level of detail. I
also assume the users weren't told whether they were looking at the genuine
results or not.

Given that, there's nothing at all surprising about that statement you
highlighted. In fact, unless Google were willing to actually run the study
themselves, randomly showing either their current results or results sorted
purely by "relevance", and gathering the real clickthrough rates, then what
the article describes here seems like the only real reasonable way to do this.

~~~
downandout
_One screenshot showed Google 's results, while another shows search results
that were purportedly sorted purely by relevance._

Therein lies the problem. They don't know how Google would order the results
in the hypothetical scenario where results didn't include those that make
Google money. This goes beyond not having clickthrough data; what they were
showing to their test users was simply a not-very-good, human interpretation
of what they presume the Google algorithm might return in a nonexistent
situation. Unsurprisingly, these fictional search results had worse CTR's than
were achieved by displaying presumably custom-crafted Yelp results.

The term _relevance_ is unique to each search algorithm, and these researchers
did not have the data to determine what the relevance would otherwise be for
Google. This report essentially says "if we ran Google, we could get better
CTR's by showing Yelp results". That does not prove consumer harm.

~~~
eridius
> _They don 't know how Google would order the results in the hypothetical
> scenario where results didn't include those that make Google money._

They kind of do. Google's very public about most of how they sort results, and
the article said they used Google's "algorithm" to rank them. I know Google
used to even expose the underlying PageRank number for various results, but I
don't know if they still do so today (if so, then it becomes even easier to
sort them the way you think Google would).

> _what they were showing to their test users was simply a not-very-good,
> human interpretation of what they presume the Google algorithm might return
> in a nonexistent situation_

You're making a very bad assumption here. From the article:

> _One set of users saw a page reflecting results currently displayed by
> Google_

Based on that it sure sounds like they actually performed various searches on
Google, screenshotted the results, then fabricated a second set of results
that reflects what they believed Google would have shown if it sorted purely
by "relevance" instead of prioritizing its own services. And these fabricated
results had significantly higher CTRs than the genuine results from Google.

> _The term_ relevance _is unique to each search algorithm, and these
> researchers did not have the data to determine what the relevance would
> otherwise be for Google._

Sure they do. The way Google ranks search results is quite public and has been
so from the beginning. What isn't public is all the various tweaks they
perform to the results, such as removing what they believe to be spam or
content farms, as well as the biasing they do in favor of their own services.
But the core algorithm is public.

> _This report essentially says "if we ran Google, we could get better CTR's
> by showing Yelp results". That does not prove consumer harm._

The report says that they believed that Google was suppressing results from
"third-party review sites such as Yelp and TripAdvisor" (not just Yelp) that
it would have otherwise deemed to be relevant, in order to prioritize Google's
own services, and that this artificial adjustment of the search results is
harming consumers because it's yielding results that are less useful to
consumers (as measured by CTR). Which is a bit different than what you're
saying.

And this actually makes sense to me. When I search on DuckDuckGo for places in
my city, I often get yelp results near or at the top. So they're obviously
deemed relevant by the unbiased ranking that DDG uses. And I often get Yelp
results if I search on Google as well. So obviously Google thinks Yelp is
relevant for many searches. So if I perform a search that Yelp has relevant
results for, and Google isn't showing me those results but instead shows me
e.g. a Google+ business listing, then it seems quite reasonable to claim that
Google is artificially suppressing the Yelp results in that case, and to
further claim that this is causing consumer harm because the Yelp results were
more relevant than what Google chose to show instead.

------
sytelus
This is basically study done by Yelp employees saying that Google results for
local intent queries include more from Google+ than on Yelp and people
participated in their study didn't liked that. Well...

Yelp obviously wants to monetize their (ummm, sorry, _users_ content) content.
I would think company like Google wouldn't have much problem paying Yelp but
probably their demands are too much (just my guess). So Google showed them the
door and decided to do their own thing and now Yelp has problem with that.

For companies like Yelp, I would suggest they respect users intent and rights.
Millions of users have sweated on to create all these content on Yelp. They
shouldn't be building a giant walled garden around it. Instead they should
just make all data available with a fair license to anyone who wants to use
it. When more companies like Apple, Samsung etc buys their licences, it might
be overall better for them than to get one huge deal out of one company. May
be they should even make their data available for free for research (they did
made small part available, but I'm saying why not all of it?). When the
community works on this data, lot of cool ideas can spark which again benefit
them to get more licensing deals and would make it worth for users who sweated
out to create all this content.

------
themeekforgotpw
If you aren't worried about the customer case you need be aware of nation
state interest in partnering with search engines and content curators - not
just to censor - but to deceive and manipulate. The leaked US Special Forces
document on Wikileaks does not mentioned Google but does talk about putting
adversaries in an information bubble that will make it look like they've lost
public support for their cause. They expand this bubble around not only the
adversary, but their family and close friends.

The way the US military works in large and increasing part is that it works
with US corporations to achieve these effects. Given the complaints by other
countries of weaponization of Google/Facebook and studies done on using search
engine results to manipulate voting we should keep our eyes peeled that Google
continues to 'not be evil'.

~~~
secfirstmd
Interesting...Which one exactly? Source?

~~~
themeekforgotpw
If you are asking about vote manipulation:

[https://www.google.com/search?q=search+engine+manipulate+vot...](https://www.google.com/search?q=search+engine+manipulate+votes)

[https://www.google.com/search?q=Search+Engine+Manipulation+E...](https://www.google.com/search?q=Search+Engine+Manipulation+Effect)

[https://www.google.com/search?q=Lok+Sabha+election+research](https://www.google.com/search?q=Lok+Sabha+election+research)

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4004139/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4004139/)

~~~
secfirstmd
Thanks. And the document about US SF units doing this?

~~~
themeekforgotpw
[https://wikileaks.org/wiki/US_Special_Forces_counterinsurgen...](https://wikileaks.org/wiki/US_Special_Forces_counterinsurgency_manual_analysis)

~~~
secfirstmd
Thanks!

------
hayksaakian
This article is awfully vague on the details of the problem.

On what basis do they decide whether results are harmful or helpful?

~~~
eridius
I think it's this:

> _The study’s authors—[…]—found that users were 45% more likely to click on
> results that were ranked purely by relevance, rather than as Google ranks
> them now, with its own services displayed prominently._

If users are clicking on results 45% less often, that suggests they aren't
getting results that are as useful to them. From that it seems fairly
reasonable to claim that this is harmful to consumers.

------
acconrad
Let us assume it does skew search results. Two things we also know are true:

* Google is a publicly-owned _corporation_. They have every right, as they own the platform, to skew it as they see fit. They aren't _supposed_ to be unbiased, if you wanted that you'd have to have a completely open-source search engine to make this argument credible.

* Just because they own the lion's share of the search market does _not_ mean you have to use them. Don't like Google? Use DuckDuckGo. Use Bing. Use AskJeeves! Are people using Google at gunpoint? Certainly not.

This whole things feels weak - isn't it obvious that Google is owned by a
corporation with it's own self-interest agenda, and by using that service, you
are bound to the skew it may or may not put in it's searches / ads /
everything?

~~~
subpopular
Totally agree here. Google is not a public utility where everyone has the
right to be represented equally or fairly.

------
eridius
Can anyone speak as to how likely it is for this study to be unbiased? It was
sponsored by Yelp, and Yelp obviously has an agenda to push. The fact that it
was sponsored by them does not necessarily mean that the study was influenced,
but it does raise that possibility.

~~~
ocdtrekkie
It's reasonable to take it with a grain of salt. Particularly to give a very
suspicious eye to their research methodologies.

------
googtardsz
Don't be (harmfully) evil?

------
bitmapbrother
I thought Tim Wu had integrity. Looks like a paycheck from Yelp was all it
took.

~~~
pasta_2
lol

It couldn't possibly be that Google did anything wrong.

~~~
bitmapbrother
Yelp is desperate. Why not do a proper independent study instead of paying
people to say what you want to hear?

