Microsoft sent out a high level research team to a company I worked at to collect data and feed back on Outlook. During my interview I asked the lead researcher why he's going through all this trouble to collect data, when he uses the same product himself, and can pay attention to how he's using it and see the disconnects between the use cases and the UI directly? It was a fair question, but I don't recall him having a good answer, other than jotting a note and agreeing. My follow up question was surly there's innovation in Microsoft around UIs, why not interview those folks? To which he and his team laughed and said you don't want any of those folks ideas. Amusingly honest, which I didn't expect.
It's very difficult to switch to a user's perception when you've spent a lot of time working on a product. Even if you didn't, being a professional in the field (game design, software development, even being an audio engineer while listening to music) really distorts your vision — so, if you want to predict the actual user/consumer's reaction, you'll better ask them.
This mentality tends to protrude in a lot of non-software related fields as well. There are many times for example where I question the layout of the instruments in my car or why my local grocery store stocks items the way it does. When you work with software/ui a lot, you tend to develop an internal organizational logic that's very different from normal users. This influence becomes quite noticeable when you have to respond to unfamiliar tasks.
The simple answer to your question is that you(the researcher or dogfooder) are not your users.
I'd say most people overestimate how much their own personal experiences generalize (to a meaningful n) or overestimate their ability to empathize with users. Even user researchers.
> During my interview I asked the lead researcher why he's going through all this trouble to collect data, when he uses the same product himself, and can pay attention to how he's using it and see the disconnects between the use cases and the UI directly? It was a fair question, but I don't recall him having a good answer, other than jotting a note and agreeing.
I'm surprised he didn't have a better answer. Experienced users cannot intuit the problems that new users will have, and the ones who think they can are the reason we have bad production interfaces. If you're used to pressing Control-Alt-Space, F, Shift-G to perform a common function, then that seems easy to you, and pressing F1 instead seems hard because it's not wired into your muscle memory. The only good way to find snags in your UI is to ask novice users in the field.
