

How to find great participants for your startup's user study - kowitz
http://www.designstaff.org/articles/recruiting-how-to-find-great-participants-for-your-user-study-2012-02-22.html

======
kowitz
I hear from startups all the time that they're struggling to find participants
for user studies. You can only recruit your friends or hang out a the local
Starbucks a few times before you start wishing for better participants, and
better data.

Michael outlines the process here that I've seen work well for dozens of user
studies that we've done together. Hopefully it'll work for you too.

------
yaliceme
I think this article's advice applies best to an established company looking
to improve an existing product for which you already have a user base that you
understand well (i.e. Google developers testing Priority Inbox for Gmail,
which is the article's main example).

I would question the value of this advice for startups, particularly very
early-stage startups. If you are idea-stage/pre-launch/pre-seed, I think this
process is actively harmful, because it encourages the bias toward over-
planning and under-doing that most of us already have (knowingly or not).
Defining criteria, writing screeners, and scheduling testers all _feels_ very
productive, but in many cases, it is actually a form of procrastination --
avoiding that scary moment when you must expose your project to a true
outsider's judgement.

Margolis writes: "Startups often point to recruiting users as one of the
biggest reasons they’re not regularly talking to their users." I would submit
that there is NEVER a good excuse to not talk to users, including "we don't
have any users yet" and "we're not quite sure yet who our target user actually
is."

The solution to this difficulty is not a 3-step process of screening,
recruiting and scheduling testers. It's much simpler, though perhaps more
difficult: swallow your shyness and go talk to people as many people as you
can. Actually talk, face-to-face, not through a survey. Give your elevator
pitch to everyone and anyone, every day, even if it's just to a stranger on
the bus or random people at your nearest university campus. You will learn a
lot more a lot faster than if you dither around with a more "systematic"
process.

When I first started working on one particular startup idea for an
entrepreneurship class, my partners and I spent a few days early on designing
a survey that we were very excited about; it was going to help us understand
people's habits, behavior, and preferences with respect to the service we were
going to provide. The instructor for that class was a serial entrepreneur; I
showed our survey to him for critique during one of my first office-hours
meetings with him. He absolutely tore me apart (rightfully so, in retrospect).
He told us we had no business making surveys at all, this early in the
process, and he refused to meet us again until we'd talked to at least 50
strangers about our idea. So I did just that, and it was one of the most
formative and eye-opening experiences I'd had up to that point.

I'm not saying that the process described in the article is _always_ a form of
procrastination, and certainly Google is beyond the point where it's efficient
to test by interrupting random strangers on the engineering quad. I'm just
saying that most startups are, almost by definition, not at a stage where
"user studies" is a productive concept.

~~~
mmargolis
Thanks so much for taking the time to read and comment on my article. I love
the feedback and discussion!

It’s true that for many products, some feedback from random people on the bus
or street can be better than no feedback. However, user studies accelerate the
learning cycle for startups. As I’ve seen over and over with many startups,
companies can more quickly learn much more by talking to the right people
rather than just button-holing 50 random strangers. And when building products
to solve problems for specific users (such as sys admins, bicyclists,
travelers, kids, English-language learners, people with email overload, etc.),
it seems a little risky not to talk to those users.

