

Ask HN: Any good tips for writing/sending surveys? - augustflanagan

I'm writing a survey that I plan to send out to ~10,000 people in the next few days. They are all members of www.lenguajero.com and many of them are quite active on the site. While we get a lot of feedback via email we have never sent a survey to our members, and I'm a little unsure how to proceed.<p>The goal of the survey is to find out three things: 1. How useful is our website/how active are you on the site?  2. What features would you like to see added/improved.  3. Are there any features (existent or non-existent) that you would pay to use?<p>Does anyone have any suggestions for how to write a survey like this (i.e. how should the above questions be asked, and what other questions should be included in the survey), or can anyone recommend some good reading material about sending surveys?<p>Also, I'm not sure how relevant this is, but half our members are native English speakers and the other half are native Spanish speakers.  Each member will receive the survey in his/her native language.<p>Thanks HN!
======
Travis
I work in epidemiology, specifically sending surveys to huge cohorts. This is
what we've found through a lit review (sorry, don't have original sources but
they're out there on PubMed).

\- incentives are huge. But know if you're asking someone to complete it for a
social benefit (e.g. our health surveys) or for your profit. If you're doing a
"feel good about helping others" thing, don't offer things that can offend.

\- make sure people know ahead of time what kind of questions and length of
time to expect (properly set expectations)

\- try to "group" similar questions. This helps people's brains load different
topics, so their responses are easier (fewer context shifts)

\- be aware that everything you do (colors, question wording, etc.) can
significantly influence peoples' responses. You must be consistent with
everyone who takes the survey (as much as you can -- you can't control the
lighting in their rooms, as much as you'd like to).

\- Make sure to be neutral in your language. Don't introduce bias by writing
leading questions.

\- try to use "standard instruments", i.e. other people's surveys that were
validated and tested. Not only does this make your work easier, but you now
have (the possibility of) a reference group to compare your results

\- don't expect very many responses (this is from my experience on all
projects). People are busy. Your survey is a small part of their day

\- send the survey invite at least twice, spread by a couple of weeks (time
permitting). Most people only complete their surveys if it's a convenient time
to do so. Don't make them look back in their email -- send them a second
invite (be careful beyond that, as you may annoy people).

\- thank them very much

\- have a detailed analysis plan before you create the survey. What are you
trying to find out? Much like test driven development, knowing what you want
to answer before you collect data is HUGE.

------
noahc
1\. Make it possible for them to take the survey in less than 60 seconds.

2\. Only ask questions you actually care about (and have the power to do
something about). If someone isn't active on the website or doesn't find it
useful, they probably won't reply to the survey. Also, even if they did does
knowing this allow you to make any meaningful changes at all?

3\. With the features question, this is what I would do. Think of 3 - 8
features you think they want. Then put an other box. Have them vote on what
the most important feature they want implemented is. If someone writes
something in other give it a multiplier 3x or something like that. At the end
of the day you'll have 1 or 2 ideas that lead the way. It'll be obvious what
wins out, but don't make them think of what features they want.

4\. I think you should ask, "Would you ever pay for features on our website"
and just get a feel for if your typical website user expects to pay for stuff
online or not. If the culture to pay for stuff isn't there, then it will be a
hard go.

------
dirktheman
Make sure you offer an incentive for those who fill in the survey. Don't make
the survey too long, but put a lot of thought into the questions and execution
(sending it somewhere in the next few days to your entire user base might not
be the best idea). Ask yourself questions first: what is is exactly you want
to know, and to what extent are you willing to take action on the outcome.

~~~
Travis
This is so important I wanted to highlight it.

>Ask yourself questions first: what is is exactly you want to know, and to
what extent are you willing to take action on the outcome.

------
kellyreid
ask questions that give you answers to specific questions! dont ask how useful
it is - ask what the most useful feature is. etc

