

What to do when users tell you one thing and data tells you another? - breckognize

My startup launched 4 months ago. We have software that online retailers (our customers) place on their product pages for their shoppers (our users) to interact with. So far, we've received great feedback from our users - they love our application. We're also positively impacting our customers through increased conversion rates and reduced returns.<p>Today, though, we discovered that the sales of the retailer are negatively impacted by one particular piece of our application. That is, once users engage with this particular feature, they become less likely to purchase (though on the whole the application is a net positive). The problem is that, from collected written feedback, this is clearly users' favorite feature. It has also been the source of a lot of buzz/press for us because it's something new/cool that has never before been part of the online shopping experience.<p>So what do you do when your users are telling you they love one part of the application, but you have data that says your customers might be better served by getting rid of it? PG says to do what's best for the user. Usually I agree with this, but our users aren't the ones paying us.<p>We've started by doing user interviews about the offending feature to see if we can better understand what it is about their interaction that may be deterring them from purchasing. It's not clear, though, that there's a tweak we can make that will change their behavior without ruining what they like about the feature. What should we do?
======
middlegeek
"once users engage with this particular feature, they become less likely to
purchase"

"this is clearly users' favorite feature."

Your description is a bit vague, but here is what I am thinking. I am
wondering if these folks were not going to buy anyway, but they stop to
"engage" this feature because it has gee-whiz appeal. So basically you are
presenting non buyers with a toy and since they are not busy with the task of
buying anyway, they play with the toy.

Be careful. Correlation does not (always) imply causation.

~~~
toast76
This is what I was going to say.

Your data may be full of noise with people coming expressly to check out your
new fangled feature. This is especially true if you've been getting press
about it.

Personally, I'd interview the people who DID buy. Ask those who used it why
they used it (and what they thought of it), and those who didn't why they
didn't.

If the net effect is still positive for your client, then I don't see what the
problem is. Maybe even just give it some time to see if the situation changes
once the buzz has stopped.

------
plux
Well, I do suspect that your startup is the one you have linked from your
profile? Well, then I first want to granulate you do a damn clever idea. (The
3d model is damn sweet information)

But I can see how it can have an impact on your retailer have a negative
impact, as you are delivering information to their customers, that they
probably had to make a purchase to figure out before.

With that said, I don't see a problem with you providing this information to
the customers, but I think your retailer need to find new ways to interact
with it's customers and encourage them to buy.

I tell you what, fire a mail to mattias at plux dot se and I will make a
suggestion, for free that you can use however you want. :)

------
zacharycohn
"We've started by doing user interviews about the offending feature to see if
we can better understand what it is about their interaction that may be
deterring them from purchasing."

This. Something sounds fishy. You should collect more data and interview more
users. Are you asking the right questions? If you can't find anything about
the "offending feature" that is driving people away from sales, then there
might be some other hidden issue.

------
thewileyone
Review your data capture process. Maybe your data is telling you the wrong
information.

