
Everybody lies: how Google search reveals our darkest secrets - TuxMulder
https://www.theguardian.com/technology/2017/jul/09/everybody-lies-how-google-reveals-darkest-secrets-seth-stephens-davidowitz
======
rasmi
I purchased this book on sale at the Kindle store to get a detailed sense of
what his methodology was, and it seems he just uses Google Trends to compare
search volumes across locations combined with Google AdWords to give the data
a numerical scale and break down the data demographically. It seems
statistically dubious at best, especially the part where he makes assumptions
based on search autocomplete results (which vary based on location, IP
address, browser, OS, and other factors).

You can read more about his methodology in his papers here [1]. He wrote his
PhD thesis on "Essays Using Google Data". I can't think of a better way to do
these estimates with the limited data provided, but I'm just not convinced
it's a strong enough proxy for the conclusions he's making.

[1] [http://sethsd.com/research/](http://sethsd.com/research/)

~~~
pmarreck
> assumptions based on search autocomplete results (which vary based on
> location, IP address, browser, OS, and other factors).

I imagine you could just do the same searches in an Incognito browser window
and get results not tailored to you, nor your location, IP, etc. (Or, I would
be rather surprised if google searches were STILL tailored to some
determinable attribute of you in an incognito browser window.)

I also imagine Seth did this (but only because I would be very surprised if he
did not).

~~~
Ded7xSEoPKYNsDd
Incognito mode doesn't do anything to hide your location or IP (basically the
same thing on non-mobile). It's really just meant as a porn mode: nothing you
do is saved to disk.

~~~
graphitezepp
Good for temporary logins to services when using someone else's machine as
well.

~~~
thaumasiotes
As long as your threat model is "I'm worried I'll forget to log out, but I'm
not worried I'll forget to close all the incognito windows".

If you're worried about the foreign machine compromising your accounts,
incognito mode will do nothing to help you.

------
AndrewKemendo
This is a well understood phenomenon in the world of Garbology [1] called the
Lean-Cuisine Syndrome [2]

In one study garbologists found that people uniformly underreported the
quantity of junk food they eat, and overreport the amount of fruit and diet
soda they consume. Most people also underreport their consumption of alcohol
by 40 to 60 percent; on the other hand, heads of households regularly
exaggerate the amount of food their families consume. [3]

[1]
[https://en.wikipedia.org/wiki/Garbology](https://en.wikipedia.org/wiki/Garbology)

[2] [http://medical-
dictionary.thefreedictionary.com/Lean+Cuisine...](http://medical-
dictionary.thefreedictionary.com/Lean+Cuisine+Syndrome)

[3] [http://www.nytimes.com/1992/07/05/books/we-are-what-we-
throw...](http://www.nytimes.com/1992/07/05/books/we-are-what-we-throw-
away.html)

~~~
FabHK
Or the well known phenomenon that when people are asked how many lifetime
opposite-sex sex partners they've had, the average reported by men is about
twice the average reported by women (though, by construction, they ought to be
the same - just imagine a bipartite graph (undirected!) and count the lines
going from left to right and compare with the number of lines going from right
to left).

See eg. David Spiegelhalter, _Sex by Numbers: What Statistics Can Tell Us
About Sexual Behaviour,_ chapter 3.

------
barrkel
Do you recall when you last read in the newspapers about how someone convicted
of murder has their searches used as evidence against them?

What comes to my mind when I read this is what the base rate of suspicious
searches is. If 10% of days include at least one suspicious search in the base
population, the value of such a search as evidence of malice is greatly
reduced. But most people don't consider base rates.

On the topic of the differences in parents' concern for their boys vs their
girls, for genius vs overweightedness, there's a simple rationale: for future
career and relationship success, a boy's intelligence and a girl's looks are
relatively more important than vice versa. It's not right or fair, but it's
believed to be true, and that's what makes people search for it.

------
tyrw
"When we lecture angry people, the search data implies that their fury can
grow. But subtly provoking people’s curiosity, giving new information, and
offering new images of the group that is stoking their rage may turn their
thoughts in different, more positive directions."

This was the most interesting takeaway for me. It's one thing to know that
negative behavior is underreported on surveys, but much more valuable to draw
potential conclusions about how the situation might be improved.

------
cko
I'm not sure that counting gay porn searches (example they gave in the
article) is a reliable indicator of whether the seeker is gay. This may be TMI
for HN but once in a while I (a straight male with zero attraction to men
ever) have searched for and "enjoyed" gay porn.

I was curious about this phenomenon and found many forum posts where straight
guys say they've watched gay porn.

~~~
rocqua
Might that mean that you are bi? I mean, if you like watching it you might
enjoy doing it.

If your first response to that suggestion was visceral, maybe you actually
have some aversion to the idea that you are. It makes sense, the world is
easier for straight people, but it might bear closer examination.

~~~
pmarreck
This does not necessarily follow. I have (ahem, my username is not anonymous
but I don't think this is controversial anymore) enjoyed porn of a nature that
I wouldn't necessarily participate in. You can also enjoy porn that is
completely fantastical and thus impossible to participate in, such as the
infamous "hentai", or (I will never understand this) that "my little pony"
porn genre, or from an unrealistic (or is it... _fantastically_ realistic?)
flash game such as "Super Deepthroat" (introduced a while back to me by a
female Redditor who claimed she could ONLY orgasm to it, perhaps
surprisingly), and if this article is to be believed, human women are
generally turned on by ANY sexual expression, even if they claim not to be,
which would ostensibly include scenarios they would be unwilling to
participate in:
[http://www.nytimes.com/2008/06/12/fashion/12bisex.html](http://www.nytimes.com/2008/06/12/fashion/12bisex.html)

(To any future employers, friends, clients, interlopers and all others who
encounter this after getting to know me... None of this should terribly
surprise you. ;) )

~~~
rocqua
Obviously, it does not follow, but it seems likely that there is a
correlation. I think it bears examination because knowing yourself is
generally a good thing.

I say this because I too enjoy some porn that I thing I wouldn't want to
participate in. I found it rather productive to consider what this means about
me.

------
andreasgonewild
How about Google revealing some of their own darkest secrets instead, that
would be interesting. There are clear incentives for the ruling powers to make
everyone believe we're all as messed up in our heads as they are, makes us
more forgiving and accepting of whatever leaks out from their filthy games.

------
gregfjohnson
I use google search all day every day. On those occasions when I want to do a
search and not have it end up in advertiser databases, I switch to
duckduckgo.com.

Do you HN'ers generally consider DuckDuckGo to be trustworthy for search
queries of a private nature?

~~~
tux1968
I personally take them at their word about privacy -- there doesn't seem to be
much choice to do otherwise. And it's easy to add !g to a search to send it
over to google when better search results are required. Actually it's really
convenient to be able to do !gm for google maps, !wa for Wolfram Alpha or use
any number of other bang commands.

[https://duckduckgo.com/bang](https://duckduckgo.com/bang)

------
Spooky23
There is a PR fluff piece about this book on HN every week. -1 spam.

In the US version of this article had a big focus on apparently secret
attraction for overweight women. I wonder what the "darkest secret" is that
pushed the author to focus on whether people are gay or not in the U.K.?

------
alexmat
The article suggests that racist jokes are a problem to be solved, but racist
jokes serve a noble purpose according to Slavoj Zizek:
[https://www.youtube.com/watch?v=IISMr5OMceg](https://www.youtube.com/watch?v=IISMr5OMceg)

~~~
Graphon1
Can't bear to watch that guy; the constant face & nose touches are very
distracting.

~~~
mwilliaams
Wow you're right. I'm just listening to him (although that's hard too with the
accent and lisp or whatever that is)

------
newtem0
What a crock of utter and complete nonsense. If you isolated the simple
statements of fact then this article would still be a long and enjoyable read,
but instead i am assaulted with corresponding insane ranting; accusations
against huge swaths of people that are totally unsupported. So nobody googles
if their daughter is smart? Maybe its because parents are worried that looks
and not intelligence are important for cohesing in school and not getting
bullied, for girls. Maybe the same for boys, but with social quickness and
intelligence. Maybe they are responding to the behaviour of the kids
themselves, who might openly worry about being smart or ugly; perhaps girls
and boys have their own tendancies about which one worries them because of the
small fact that they are differnt biologically, and that visual attractiveness
is much more important for men? Not a single alternative is explored by this
author and it should be no surprise because when you zoom out and look at the
whole picture of this article its obvious that this guy is simply using these
issues as a tool for popularity and self advancement. Im not saying it
shouldnt appear on hn because the phenom of using these issues for personal
gain and the death of journalism is in itself an intellectual curiosity. But i
will die before letting this shit go uncommented upon.

~~~
pjreddie
> accusations against huge swaths of people that are totally unsupported.

The search data is meant to support the accusations. A main point is that
other methods of supporting such statements (like surveys) are unreliable.

> So nobody googles if their daughter is smart?

The article doesn't say that, it says parents are about twice as likely to
google if their son is smart vs daughter.

> Maybe its because parents are worried that looks and not intelligence are
> important for cohesing in school and not getting bullied, for girls.

This seems like exactly the problem the author wants to bring up and address.
Society values appearance above intelligence in women. This is apparently also
true in how parents see their children.

The author doesn't draw conclusions about this prejudice, he is merely
remarking that it exists.

~~~
thaumasiotes
> Society values appearance above intelligence in women. This is apparently
> also true in how parents see their children.

Sure, parents who are interested in their children's well-being will also be
interested in what society thinks of their children.

This data seems like a weird source for the claim, though; I'll bet the people
searching the internet for "is my daughter smart" significantly outnumber the
people searching for "is my daughter pretty".

~~~
pjreddie
I'll take you up on that bet.

[http://imgur.com/a/SqCKL](http://imgur.com/a/SqCKL)

~~~
thaumasiotes
"is my daughter smart" and "is my daughter pretty" both show up as 0 in google
trends (as I search today). What does "daughter pretty" include? I would
predict massive public interest in questions of the form "is Reese
Witherspoon's daughter pretty?"

~~~
pjreddie
Yeah, they don't show up with a zero, it just says not enough data to show so
hard to draw any conclusions. I agree this is an imperfect proxy and it's not
definitive but I'd still absolutely take you up on that bet (conditioned on
there being some way of actually resolving it).

But maybe the real question is, why aren't people asking "Is Reese
Witherspoon's daughter smart?"

~~~
thaumasiotes
In specific, obviously because Reese Witherspoon is known for being pretty.
I'd also bet that "Is Bill Gates's daughter smart?" is a more popular search
than "Is Reese Witherspoon's daughter smart?"

------
Godzillaz
Not a single alternative is explored by this author and it should be no
surprise because when you zoom out and look at the whole picture of this
article its obvious that this guy is simply using these issues as a tool for
popularity and self advancement. For example on those occasions when I want to
do a search and not have it end up in advertiser databases, I switch to
duckduckgo.

