

Don't Trust Your Mother. Small Usability Tests Are Worse Than No Usability Tests - phlsa
http://www.envis-precisely.com/blog/dont-trust-your-mother/

======
mdc
I've been involved in two types of usability studies. The first are academic,
rigorous, well-designed experiments testing readability, visual search, or
other issues in web site design. These were designed to provide statistically
significant results. The second are small focus groups used to test,
predominantly, GUI design elements.

This article claims the second kind of test is worthless and misleading, but I
think that's ignoring obvious benefits. No, you don't get statistically
significant results, but in most cases you're not aiming for some result like
"85% of users prefer the blue button". More often, you want to watch a handful
of people use your interface and a result like "2 out of 5 were really
confused by the menu structure, so we should rethink that". That is, your
useful data comes from watching the process, not recording the result.

Rigorously-designed experiments which provide the potential for statistically
significant results are great and they definitely have a place. But they're
not the only tool in the box.

