
Your Startup’s Office Is Missing a Room - tlongren
http://tomtunguz.com/usability-lab
======
soneca
I started reading this as "wow, they live in a very different world in SV
where you can advise startups to set up a room like this". I am in Brazil and
here there are lots of startups with founders working at home, not even an
office, much less a usability room. At the end, at least the author sugests
that this is for post-series A.

But then I remembered, I did this myself! At home. With my father. In a much
leaner way.

I am a sole founder, working at home, while my wife pay for our bills (all
that to say, I have no money to invest). My product is sort of "online pills
dispenser as a service", a SaaS B2C helping elderly people organize their
medicine. And my father is just my target (actually, I started all this
thinking of solving his problem). When created the first version I went to his
home and let him use the site. I just typed the URL and then just kept
watching. He pains attention to details, so I left, one hour and a half later,
with literally more than 50 notes of changes that I should made. From small
design and copywrite details, to whole new features.

And you don't need webcam, microphones and a soundproof glass. This is the
case if you have bosses for whom you must report. If you are the founder, just
you, there, taking notes(and asking the right questions) while the user
experiment your product, it is enough.

If your friends are the target, your house is fine. If you must talk to
strangers, a public computer is enough. Even Eric Ries on his book talks about
customers going to their office to do this - not a whole "usability room".
Sometimes I think funded startups spend their investor's money just for the
sake of it.

~~~
potatolicious
I agree. This sort of usability testing is valuable - more valuable than simpy
A/B testing in a vacuum - but it's important to remember that this is
extremely qualitative and unscientific.

Throwing in the webcams, eye-tracking, or what-haves-you IMO incorrect lends
an air of specificity and scientific-ness that isn't warranted.

Before we launched our last product update we brought people in to use the
app. It was very much as you described - we invited them to come into the
office where our PM sat with them, watch them used the app, and asked good
questions.

The much more subtle behaviors are IMO not determinable via pseudoscientific
things like watching facial expressions, eye movement, heart-rate, or whatever
these things are supposed to measure. Best A/B test for the rest.

~~~
soneca
I totally agree. This is a qualitative experiment. You try to generate
insights, assumptions, hypothesis from it. Then after that you test these
insights more scientifically.

Including this, very well put, pseudoscientific things, on the qualitative
phase must do more harm than good. Maybe they act as an excuse not to really
test the assumptions.

------
100k
You can get a ton of value out of usability tests without going all-in on a
usability lab. Buy screenflow and a USB microphone to record sessions, and
then recruit people for $50 a pop to come in and go through your product.

Steve Krug calls it "Lost Our Lease Usability Testing" and explains how to do
it in Don't Make Me Think. You can download the relevant chapters here:
[http://www.sensible.com/downloads-
dmmt.html](http://www.sensible.com/downloads-dmmt.html)

We did this at a company I worked at a few years ago and it made us completely
change how we were building the product because our target users got nothing
out of it (I talked a bit about this a RubyFringe:
[http://www.infoq.com/presentations/francl-testing-
overrated](http://www.infoq.com/presentations/francl-testing-overrated))

~~~
ttunguz
I don't think a usability lab is that expensive. A few webcams, a mike, plus a
laptop and mobile phone is all you need. That probably totals to at most $3k.

Both kinds of feedback, lab and cafe, are useful. They each provide a
different kind of feedback, from a focused and unfocused user so they aren't
mutually exclusive. Rather, both are worthwhile.

~~~
ricardobeat
Keeping an extra room costs way more than that. It doesn't make any sense for
most companies to have a dedicated lab, you're not going to run usability
tests all day, every day.

Plenty of market research companies offer rooms like this for rent.

~~~
rdouble
It doesn't cost that much, really. The salary of the guy running the lab is
the largest expense by far. At a startup I worked at a few years ago, they had
a lab running 40 hours a week. It would have cost about 10 times as much to
have a market research company do this for us if it was even possible.

------
JoblessWonder
When I worked for Citrix Online I took part in internal usability studies. It
was great to be able to sit with a project manager and explain what made sense
to me as a user and what didn't. It was also thrilling to see a suggestion I
made make it into the final product a year after I left the company!

------
seiji
Is it the perf room? Please tell me it's the perf room.

(Relevant anecdote: where I used to work had a makeshift UI rig with webcams
and laptops. It could be dragged into any conference room for bringing in end-
users (teenagers) to try out new games/experiences/interfaces in progress. How
do yo find end users? Offer $5 to $20 gift cards on craigslist for their time
if your product is broad enough.)

~~~
ttunguz
This comment made me laugh -> the perf room!

------
garrickvanburen
In my 15+ years experience, I've found 1-on-1 conversations at a shared table
(sometimes even at a coffeeshop over coffee) to be provide just as actionable
and informative feedback as those of a lab environment. Add in the
dramatically lower price tag and the honesty of a human-to-human conversation
and I head to the coffeeshop everytime. I do believe there are one or two
scenarios where the lab is preferred. I try to steer away from those projects.

~~~
notatoad
>the honesty of a human-to-human conversation

i think that the _dishonesty_ of human-to-human conversation is exactly the
problem that usability labs are trying to solve. People are way less likely to
complain about your product to your face, they'll keep insisting it's good and
you'll have to work harder to drag the problems out of them.

------
carsongross
And yet, and yet... the new Gmail.

~~~
notatoad
...passed through extensive usability testing. just because you don't like it,
doesn't mean it sucks. that's pretty much the entire point of actually doing
testing rather than relying on some guy's personal opinion.

~~~
carsongross
Ah, but I, _and my friends_ didn't like it.

QED, right? ;)

------
imroot
There's someone in Cincinnati who is doing this in a much, much different
fashion -- they're buying drinks in return for 1-5 minutes of interaction with
an iPhone app. It's a really unique way of getting instant feedback on his
app. I've noticed that he's semi-responsive to his changes -- he'll be back a
few days later with a new build, and a sbux gift card, looking for more input
on his changes.

------
vadivlkumar
One of the problem we face now on our product is same thing and we also do a
similar excercise to improve product usability. But what we found was most of
the suggestions were purely subjective and biased. Picking up right audience
is the key. For example, instead of calling a CEO who is going to buy my
product to evaluate i would rather invite someone who will use this product
for living.

------
Zimahl
Not to disparage the idea but this can be expensive to set up and time
consuming to use. Also, it can take a while to get the process down and/or
have someone who can properly interpret what the user is actually doing.

I have found that in early stages of a startup that one or more subject-matter
experts who are hopefully involved with the space can give much more direct
feedback in the early stages.

