
What My User Survey Taught Me - stakent
http://www.kalzumeus.com/2010/02/07/what-my-user-survey-taught-me/
======
tptacek
We've been very happy with surveys at Matasano; we use Formspring, not Wufoo,
we drive them from Twitter (ha! Patrick), incentivize them with posters and
Matasano refrigerator magnets, and are using them to generate content. Every
survey we do is good for a measurable number of new customer prospects.

The thing I want to mention though is that if you're out of ideas for media
relations, and you run a survey often enough with general enough questions,
you can take the aggregate results of the survey and shop them to reporters
for trend stories. It is one of the older PR tricks in the book.

------
patio11
This is a followup to a post I did on survey incentivization, covered on HN
here: <http://news.ycombinator.com/item?id=1058669>

Short version: survey incentivization worked very, veeeeeeerrrry well for
increasing response rates. The survey itself got me actionable insights, and
when I say that, I mean I did eight commits today based on them. I'd really
recommend you try one for your business if you haven't already.

~~~
cool-RR
I appreciate that you said you aren't angry at people who complain that your
product isn't given for free. I hate it when developers are angry at people
who want their software to be free.

------
qjz
"The sites that I advertise on do not optimize for their user experience
because if their website is better than my textual ad, they don’t get paid."

Eureka! Offer a free web-based product so lame that it drives visitors to
click the ads of competitors. Even I can do that!

~~~
revorad
Think plentyoffish.com.

------
paraschopra
One aspect that I always worry about surveys is that there is tremendous self-
selection when it comes to who chooses to send time taking the survey. I also
have a survey put up at Visual Website Optimizer and never for once I got a
negative feedback. That may be because the product is awesome but more likely
it is because the users to choose to invest time answering the survey already
like the product and want to pay back to me in terms of their time.

What is more valuable to me is feedback from people who hated the product, but
irked them and what can I fix. Getting those people to fill the survey is, not
surprisingly, really hard.

Patrick, what is your opinion on this? Do you worry about self-selection too?

~~~
patio11
Self selection worries me a little bit, but I try to focus my worrying on
things I can change rather than things I cannot.

I got a lot more helpful negative feedback by surveying users than I do by my
typical approach of sitting by the email box waiting for someone to complain.
That is enough reason to do the survey for me.

Of course, test and measure to supplement the feedback people give you with
data on what they actually do. If they say "DON'T CHANGE A THING!" and your
data says only 5% of users come back, time to fix.

------
callmeed
I just did a year-end customer survey. I used a Google spreadsheet/form.

I considered incentivising it, but chose not too in the end.

 _"Incidentally, you might think that you’d get lower quality feedback from
incentivized users. I did not get that impression from reading the results,
but I can’t reduce that to a simple statistical measure."_

I'm curious what gave you that impression–just the fact that people seemed to
give honest (i.e. good and bad) feedback?

Also, I'm curious what you think about other incentives–like drawing names
from participants and giving prizes? (for situations where you can't simply
give free product)

~~~
patio11
_I'm curious what gave you that impression–just the fact that people seemed to
give honest (i.e. good and bad) feedback?_

Folks answered the questions despite any technical requirement forcing them
to, and many of the responses looked like they had thought put in them. (Of
course, many users from both groups gave answers which were not triumphs of
well-expressed English but it didn't seem like folks were mostly typing
asdfasdfadf to get their freebie.)

I like guaranteed prizes over random prizes because my experience e.g. in
teaching is that surety is a powerful motivator/instructional tool. (Then
again there is also the WoW model of random rewards over repeated
interactions, but this is more of a one-off.)

