
Google Consumer Survey - iag
http://www.google.com/insights/consumersurveys/home
======
RobAley
Don't use this if you are going to spend money based on the outcome (or do
anything else important with it).

People do not have a vested interest in completing the questions honestly or
on the outcome. Their motivation is to get to the premium content, and
although there will be some that are honest, most people are trying to get
"free" content and have the most motivation towards not giving value (money or
answers that betray their privacy/thoughts) back.

My day job is primarily based on analysing survey results, and I can tell you
that the only way to get decent usable results from "paying" people is to make
sure that they have a vested interest in the results of the survey and that
the payment is a (nice) side-effect of completing the survey and not the goal.

Edit: Of course, this type of survey can be useful if you need to "validate"
something that otherwise you can't (i.e. to falsify results). Simply choose
multiple choice questions and stack the answers in the right order (people
usually click in certain pattern when doing it at "random"). Keep repeating or
extending your survey until you have the right ratio of participants in favour
of your preposition, then call it a day and publish!

~~~
michaelt
Can you give me an example of how people having a vested interest in the
results of a survey would improve the results?

I always thought that if you have a question (like "should the government fund
more particle physics") and you ask people with a vested interest (particle
physicists) you'll get an answer aligned with their interest ("absolutely,
extremely important") in which case why do a survey at all when you know what
answers they're going to give?

~~~
RobAley
You do indeed get an answer aligned with their interest, which is often useful
(not perhaps in your example), as that is generally the point of the survey
(to find out what your respondents think).

You need to pick your respondent cohort carefully if you are trying to ask a
question of opinion (particularly more so than when asking a question of fact)
to eliminate bias in your cohort. So in your example, rather than surveying
particle physicists, you would look for another group of interested parties
(e.g. a range of tax payers or non-specialist funding bodies) who have a
vested interest (in this case in how their money is spent).

My work, for instance, revolves around surveying doctors on their experiences
of medical training and career needs/aspirations. The vested interest of our
participants is hopefully clear (influence on their training/career/work
environment). We ask a wide range of factual and opinion based questions,
eliminate biases (or segment responses on biases) due to various factors (e.g.
location, age, gender etc.) and report our findings in the context of our
cohort, i.e. "these are the experiences and opinions of doctors".

In short, you are right, you need to consider the makeup of your cohort and
also how you report your findings in respect of that. Having a vested interest
doesn't necessarily mean being biased, and sometimes that bias is what the
survey aims to report on. In a survey like the google ones, that bias is
money/free content, and thats all you will discover.

------
franze
the thing is with tools nobody knows about is, that google regularly kills
them. i.e.: <http://trends.google.com/websites?q=> google trends for websites

this was the unbelievable useful tool to get competitive data (traffic,
queries) about other websites. it was not perfect, the data was sometimes
months late. but it was good enough to compare websites with each other.

now it's gone. probably it didn't fit into google strategy. probably someone
at google decided that "we are not in the business of providing competitive
intelligence data of websites" (i guess that, as the google adplanner was
stripped of lots of website data in the same timeframe )

the same thing with the product mentioned above. it will be killed. maybe not
this year, maybe not next year, but sure as hell sometime.

~~~
melg
Instead of "Trends for Websites", you can use "Google Ad Planner "
<https://www.google.com/adplanner> which has more detailed traffic statistics.

~~~
franze
nope, not anymore (as mentioned above) i.e.:
[https://www.google.com/adplanner/#siteSearch?uid=twitter.com...](https://www.google.com/adplanner/#siteSearch?uid=twitter.com&geo=US&lp=false)
killed the product (for most websites on the web), too.

------
mansoor-s
I've seen these kinds of things in the past. I can say first hand that I for
one do not read the questions... type gibberish in text fields and select
random choices in multiple choice answers. I am sure others will do the same.

I don't see how this can work. The only time I see it some-what-working is
while gathering a massive (really massive) amount of submissions for a
decision involving only two options. Maybe then... but even then it wouldn't
be possible to determine the actual percentage of users suggesting one or the
other option.

------
mtrimpe
You can email Paul McDonald to get a free voucher to try this out (see
[http://www.quora.com/Google/What-is-it-like-to-use-Google-
Co...](http://www.quora.com/Google/What-is-it-like-to-use-Google-Consumer-
Surveys))

I used it to ask how much people were willing to pay to collect video customer
reviews and a _lot_ of the responses were 11111111, 1000000 or 12345 dollars
per month though.

~~~
drharris
Never, ever make a free entry question. You'll wind up with "balls", "suck
it", etc. all the time. Predefine 3-4 set ranges, and let them perform a
single click to answer the question. Much more accurate results.

~~~
sadga
more accurate results, or more plausible-looking random results?

------
brennenHN
I used this once and was really happy with the results. I think there's a
little bit of an opportunity bias which good statisticians would have issues
with, but for some useful imperfect numbers I think it's a good solution.

Also, this is a pretty new tool.

------
interg12
This type of market research was previously only accessible to companies that
could drop 10k or more on these surveys. This tool will help entrepreneurs
validate ideas they were unable to test because it was cost prohibitive.

~~~
golovast
I spent a couple of years working for a company in a market research space.
It's an interesting business of which I knew very little before I worked
there. But the answer is that you could do a survey for a fraction of that
cost. SurveyMonkey has the same product and so do a few others.

As an aside, google launched this about 6 months ago and pretty much has
everyone in that business scared. Ultimately it all boils down to having
profiles on a lot of users and selling access to them. Google's user base (and
what they know about them) dwarfs what everyone else has combined. If they
wanted to, with a bit of effort they could own that market in 6-12 months.

~~~
interg12
I hear you on that - but that survey monkey panel is going to cost $3-$8 a
response depending on the audience you want to reach (i.e. 50 year old working
men tend to be expensive), and the results just aren't that pretty looking
like googles. I wouldn't be surprised if Facebook comes out with a similar
product - they seem to be in a better position to give users rewards to
perform MR activities, and have more data on users for additional cross tabs;'
Google only has approximate age, gender, location.

------
waterlesscloud
So for $110 I could run my own presidential election poll?

Hmmmmm.

No, wait. Targeting is 50 cents a person, not 10. So it would be about $550.
Still, though.

~~~
diego
I did, a week ago:

[http://www.google.com/insights/consumersurveys/view?survey=b...](http://www.google.com/insights/consumersurveys/view?survey=b67pd74kkvau4&question=4&filter&rw=1)

~~~
waterlesscloud
Very interesting, thank you for sharing that!

The patterns in subpopulations seem to match general expectations. Very cool.

Almost no responses in the $100k+ incomes. I wonder if that's a general
problem with the method?

~~~
gojomo
It could be a general problem. For collecting responses from high-income
individuals the method faces the double-whammy of: (1) smaller number of
potential respondents; (2) respondents are least willing to spend their time
on such surveys (or the sites the survey-wall blocks).

------
sami36
This is brand new, I saw it previewed on JLG Monday Note blog. The consumers
are going to be incentivized to take them as a form of micropayment for access
to articles behind a paywall...win-win for everybody I guess.

~~~
interg12
consumers are incentives to take the survey to see premium content - I didn't
see anything about micro payments.

~~~
sami36
Google will charge the survey initiators, take its cut, pay premium content
providers with the balance.

------
ckelly
Glad to see the interest in this new form of market research. YC S12 company
Survata.com offers a similar service. We can run multiple questions at a time,
so researchers enjoy the ability to cross-tab responses (an important ability
you lose when you run only 1 question surveys on other services). We encourage
you to check us out at Survata.com (we give all first time users a $10 coupon)
and we welcome any feedback.

------
Axsuul
Although this idea isn't new, I'm glad Google is getting into this space in
order to make it more competitive and cheaper to perform these types of
surveys.

------
arpit
This is clearly not my domain, so I am sure I am missing something, but how is
this different from creating, say, a Wufoo form to prompt users to enter
information that you need? (WuFoo seems to be $0.05 per entry)

~~~
burgreblast
If I understand correctly, Google supplies the traffic.

------
auston
The term for this is "content-locking". The opposite side of this relationship
(for publishers): <http://www.blamads.com/>

_disclaimer: friend owns that site_

------
laacz
Won't this lead to pay-walls adopting this and eventually creation of pay-per-
view non-news sites? If so, people will blindly click thgough and quality of
data won't match expectations.

~~~
Axsuul
Survey pay-walls have been around for a long time and I would imagine a lot
people already click through them blindly. I expect the price will eventually
drop further so that you'll be able to afford enough responses to get past
that margin of error.

------
netmau5
Well the paywall thing is a huge deal-killer for me although I love the a
response dashboard system they've built. For those of you conducting surveys,
what are some good alternatives?

------
tsieling
Now you, too, can test 41 shades of blue.

