> During my interview I asked the lead researcher why he's going through all this trouble to collect data, when he uses the same product himself, and can pay attention to how he's using it and see the disconnects between the use cases and the UI directly? It was a fair question, but I don't recall him having a good answer, other than jotting a note and agreeing.
I'm surprised he didn't have a better answer. Experienced users cannot intuit the problems that new users will have, and the ones who think they can are the reason we have bad production interfaces. If you're used to pressing Control-Alt-Space, F, Shift-G to perform a common function, then that seems easy to you, and pressing F1 instead seems hard because it's not wired into your muscle memory. The only good way to find snags in your UI is to ask novice users in the field.
I'm surprised he didn't have a better answer. Experienced users cannot intuit the problems that new users will have, and the ones who think they can are the reason we have bad production interfaces. If you're used to pressing Control-Alt-Space, F, Shift-G to perform a common function, then that seems easy to you, and pressing F1 instead seems hard because it's not wired into your muscle memory. The only good way to find snags in your UI is to ask novice users in the field.