

Usability Testing: the secret sauce of startups - alabut
http://refer.ly/blog/usability-testing-for-startups/

======
columbo
> Testing Mobile Apps On Drunks

I've done this before. Not a mobile app but a unit of hardware that is at bars
(think those digital jukeboxes). During the final stages of testing we mounted
one at a friend's place and set it to debug mode (so anything with a magnetic
strip passed as being a valid credit card).

After a few hours of drinking and watching friends we uncovered some
interesting usability concerns. The only problem was we never designated
someone as a sober-recorder so the notes were almost entirely useless.

~~~
alabut
That's hilarious. I should've mentioned in the section on testing drunks that
the moderator should stay at least somewhat sober :) but I'm glad you got some
insights at least.

------
ryanglasgow
Interesting article and very relevant since I've been conducting bi-weekly
usability testing sessions. I've had a lot of success using IamExec.com to
find participants for one-hour loosely scripted sessions.

Some feedback: It's hard to tell what's a link as the purple really blends in
with the black. The dichotomy between "newbie's" and non-newbie's is a bit of
a turn-off and could have been more fluidly discussed.

~~~
alabut
Great idea on using Exec - I hadn't thought of that. We've got Taskrabbit
upstairs in our building so that's an option as well.

------
AznHisoka
Wouldn't completing a list of goals contradict "not ask leading questions"?
Why not just give the app to the user and tell him/her to do his job with it?
Instead of telling him what to do.

In the end, the app is supposed to help him accomplish what they want, not
what you want them to do.

~~~
alabut
No, it's not a contradiction to have an agenda in mind and to also try not to
ask leading questions.

An example of a leading question in this context would be like asking them "do
you see the button labeled 'learn more' to your right?", which really doesn't
test anything other than the fact that they're literate and their eyes work.

Compare this to asking them "now what do you see on the page and what do you
want to do?" Maybe they'll want more information and hit the learn more
button, maybe they want info and don't see it, or maybe they want to do
something else entirely.

And sure, you can just give the app to the user and see what they do, without
writing up goals in advance. That's the unscripted method that Mark Hurst
popularized and I linked to. Here's an even longer article by him on the same
method:

[http://www.goodexperience.com/blog/2004/12/tips-on-
moderatin...](http://www.goodexperience.com/blog/2004/12/tips-on-moderating-
listening-l.php)

But that's not appropriate for every situation. Unscripted stuff is great for
getting the lay of the land and uncovering as many problems as you can, or
finding out if they'd use your app at all even if you got rid of all the
obstacles. Often times you already know about specific usability issues,
however, or you want to make sure they cover all the bases in an app with a
lot of moving pieces.

------
ananddass
What are some of the tools you use to synthesize the learning from the
usability testing?

~~~
alabut
It depends on the method of testing. If I'm doing an in-person session, I'll
record the session with Silverback, review the footage in there and make some
notes along the way, then build a quick powerpoint deck with major findings
and selected video clips.

If I'm using usertesting.com, then that stuff is all built into the app itself
and you can just send the videos around, which'll be annotated with your
feedback. Each note is a clickable timestamp that takes you straight to the
time slice you want to highlight.

Either way, it's important to not only review the test afterwards but to also
pull out clips that you can show the team, because video has such a
motivational impact and cuts through debate. It helps if you can take a few
minutes in imovie to make a "lowlights reel", where different people all run
into the same obstacle.

And if I'm doing the drunken bar testing, I just bring a witness to watch and
compare notes with, hopefully also somebody sober :)

