
Google Surveys: Know What You Are Asking - diego
http://diegobasch.com/google-surveys-know-what-you-are-asking
======
sadow
(disclaimer: I work for Google Consumer Surveys)

Here is a February 2012 study from Pew Internet, citing that 66% of the
American internet population uses social media.

[http://pewinternet.org/Commentary/2012/March/Pew-Internet-
So...](http://pewinternet.org/Commentary/2012/March/Pew-Internet-Social-
Networking-full-detail.aspx)

Our data, in addition to the survey shared on HN earlier, shows very similar
statistics.

~~~
diego
It's not exactly the same question. Wouldn't that response include much more
than Facebook and Twitter? E.g. Pinterest, LinkedIn, Reddit, Google+, a long
tail of social media sites?

~~~
sadow
It's a great point. To compare benchmarks, we'd have to consider...6 months in
behavioral changes since the Pew survey; the likelihood that long tail users
also use the main social properties; what we actually classify as a social
network; medium effects on answer bias and more.

I couldn't agree more with your points about the importance of survey design.
Our hope with GCS is to provide a real time, affordable, iterative mechanism
for conducting this type of research.

The challenges faced when benchmarking against Pew and others, is, I believe,
an indication of the value of continually collecting and updating data.

------
jey
The author's point is true of all surveys and has nothing to do with Google
Consumer Surveys in particular.

~~~
diego
That is correct. However, Google Consumer Surveys is accessible to laypeople
for a relatively low price. I would assume that when a company commissions a
Gallup Poll, Gallup ensures the poll is properly designed. On the other hand,
people without the proper knowledge could spend money on Google polls only to
draw wrong/useless conclusions.

~~~
pgeorgi
We had two professionally conducted polls in Germany by one company, asking
more-or-less the same thing (internet censorship as means to combat child
pornography), commissioned by different stakeholders (one in favor of internet
censorship, one against), with different phrasing. Each got >90% approval for
their preferred point of view.

That company took no issue with running both polls (telephone, n=1030,
"representative of people living in Germany"), and merely pointed out the
differences in the questions when asked for comment afterwards. The second
poll was mostly run to prove exactly that point.

It seems they take money for whatever poll they're asked to do, no matter how
useless. They probably even optimize it to confirm the bias of their client -
since polls are usually used for (internal) marketing purposes, I'd assume
that's usually desired, too.

I wouldn't expect Gallup to do different. That's their business.

------
nc17
Suppose I want to run a survey, and I want the responses to be as unbiased as
possible (not like in the video). How can I learn to do it "the right way" ?

~~~
malloc47
This is what I have noticed several social scientists reference when
discussing survey/questionnaire development:

[http://www.amazon.com/Asking-Questions-Definitive-
Questionna...](http://www.amazon.com/Asking-Questions-Definitive-
Questionnaire-Questionnaires/dp/0787970883/)

------
klawed
I would posit that the issue with the original post referenced by Diego is, in
fact, that the conclusion he draws is poorly stated. Reading the full post by
Jamie, it's clear that the title "34.5% of US Internet Population not using
Facebook/Twitter" should actually read 34.5% of US Internet Population not
using Facebook/Twitter _to manage their identities on third party sites._ "

~~~
furyofantares
34.5% chose the answer "No - I'm not on Facebook/Twitter", there were other
ways to answer "no" that didn't imply that you weren't on Facebook or Twitter,
and in total the poll showed 77% responding that they didn't use
Facebook/Twitter to manage their online identities with third parties.

~~~
codeka
The only way to choose "no" is "no - I don't understand how it works" or "no -
I'm scared of scams". I know how it works and I'm not scared of scams, but I
still don't use Facebook to log in to third party sites. What do I choose?

~~~
furyofantares
That's an issue with the survey, it's not clear what people in your situation
chose.

------
sunnysunday
I'm not interested in what percentage of the population uses social media. I'm
interested in increased sales/subscriptions.

It's the difference between speculating and actually making money. It's the
difference between making a survey (the beginning, perhaps, of an indirect
path to something... who knows what) and making a sale (direct path to booking
revenue).

Programmers have a particular love for indirection. And the web has a
particular love for speculation. Can we sell surveys? Yes, I think we can.

------
akldfgj
Eh, seems plausible that Jamie got >$150 worth of return for his advertising
spend on the survey.

I'm sorry, did you think that there was _science_ happening in the HN self-
promotional echo chamber?

