

Avoid Customer Feedback Before Version 1.0 - adii
http://adii.me/2011/08/avoid-pre-v1-feedback/

======
jjm
I can't seem to find myself in agreeance with what the author has written. It
might be in the way it was written (author could have been pressed for time
and compressed what he wanted to say).

How many years have we validated the effectiveness of "Get out of the
Building" already? Not talking to customers before version 1.0 is against this
highly effective principal.

Lean startup is not just 'iterate'. It's really a "huge customer feed back
loop." You iterate based on things you learned from customers.

Replacing 'tech' with restaurant might make it easier to understand. The
author is suggesting releasing food to users at a restaurant opening, prior to
actually testing whether people want that food, think the food is good, or
even if that food is appealing to the correct customer segment.

What the author and most commenters have already gleamed, is that customers
can be wrong. But they can also be right. The same principles apply to
founders as well. Here is a quote from Eric Ries:

## _Learning and executing customer feedback skills is as important as
coding._ ##

Carefully snipped from:
[http://www.startuplessonslearned.com/2010/09/visionarys-
lame...](http://www.startuplessonslearned.com/2010/09/visionarys-lament.html)
Go read the entire post. " ...

This is where data, focus groups, customer feedback, and collaborative
decision-making get their bad rap. In many cases, these activities lead to bad
outcomes: watered down vision, premature abandonment, and local maxima.

When visionaries say “but customers don’t know what they want!” they are
right. That’s the problem with false dichotomies: each side has a kernel of
truth within it. You cannot build a great product simply by obeying what
customers say they want. First of all, how do you know which customers to
listen to? And what do you do when they say contradictory things?

...

I recommend a mantra that I learned from Steve Blank: always consider your job
to find out if there is a market for the product as currently specified. Don’t
try and change the vision every time you get new data. Instead, get out of the
building and look for customers for whom your product vision is a slam-dunk
fit. If and only if, after exhaustive searching, you cannot find any customers
that fit the profile, is it time to have a serious conversation about whether
and how the vision should be modified (a pivot).

... "

Now read the post title again, "Avoid Customer Feedback Before Version 1.0".

You see, its actually fundamental to keep the feedback loop going _as early as
possible_. Even before version 1.0. This is why I don't agree with the article
author. Again, it might be because of the way the article was written (it's
always good to have more details) and trying to compress a startup in several
paragraphs is near impossible.

Your goal isn't to put out a 'complete' app. Your MVP should be focusing on
only providing a solution for the very defined simple and tiny problem which
you found. In that exercise you will have to filter through customer feedback.
Feedback includes metrics gathering whether or not people are using your
product and if people are clicking on the wrong stuff. This kind of feedback
helps you correct usability issues.

You cannot just go out and ask users "What features do you want." That is
nuts. You need to script it carefully, you need to engage and ask users about
ambient ideas, feelings, thoughts. Sometimes you also just need to let the
customer speak. You will need to develop customer skills. There 'might' be
some interesting discoveries along the way.

Eric Ries and Ash Maurya recommend about about 30 minutes for customer
interviews and the script is VERY carefully put together as to AVOID "what
features do you want".

~~~
mbesto
This is how I see the problem. Feedback comes in different forms. Giving a
potential client (or beta client) a survey that says "What can we improve?"
might not yield the results you want. Listening to a client talk about the
product, the interface, how they use, what they talk about in a day to day
conversation, will yield you insight to the real problem. If you ever listen
to PG break down proposed business ideas, he's very acute in finding real
logic and reasoning behind problems people have and the solutions people are
proposing. I believe this is both a mixture of natural ability (perhaps what
we generally consider IQ?) and experience - not some manual you can read.

Most of us techies are very analytical - which means we analyze something and
try to reproduce it. So when we read things like the lean startup or this blog
post, we think "Copy+Paste = Success". I don't think there really is a manual
for getting user feedback. I think you either get it or you don't (or you
eventually get it). The lean startup is a framework. Like any framework, it
doesn't do anything unless you understand it and know how to leverage it.

~~~
jjm
I agree and that is why I tried to frame my sentence above "Learning and
executing customer feedback skills is as important as coding".

I am a firm believer that It can be learned. In fact I think many VCs look for
entrepreneurs that are willing and able to learn these important skills.

I look at it this way as well, it is much easier to teach a hacker customer
skills than it is a business person how to code, and code well (like a good
hacker).

~~~
jpwagner
On your last remark, I've come to realize this assumption is false. Two
points: (1) In the same way that hackers have practiced thinking like hackers
all their lives, gifted business people have been practicing what they do well
since a very young age. (2) What makes it "hard" for business people to learn
to code may not be that it's hard to learn, but that the ROI isn't great
enough.

~~~
jjm
Why would the 'I' in ROI not be (in some respect) the effort required to teach
hacker-ness?

------
hashbo
Nice thought provoking article and I can empathise with it. But the title is
dangerously misleading IMHO.

All feedback is valuable. Decoding it is the entrepreneur’s job - shutting off
feedback from your user base can be a quick death march to oblivion. Hear
their pain, ignore their demands :-)

If it’s overwhelming have a private beta and gradually let people in (we’re
doing that right now at <http://hashbo.com> \- I know, I know gratuitous plug)
- listen to the first groups pain. Ease it where possible and accept the
limitations present. Move to the next. You won’t please everyone, doing so is
the pitfall I believe this article is aiming at. Encourage users to feedback,
but put it in a list you don’t read. Every now and again look at the list for
trends, for themes and reconsider if those themes are within product scope, if
they are then get on it, if not forget it.

There’s a huge middle ground between between reacting to each demand and
ignoring all feedback :-)

Personally I decode feature requests into user desires or user pain, so I both
agree with the principle of what you’re saying but feel it’s perfectly okay to
receive feature requests if you are willing to decode them :-) But to be fair,
my follow up the question is, why do you want this feature (i.e. where is the
pain :-) )

Ok 2c given :-)

------
blatherard
Rather than say avoiding customer feedback altogether, I'd say to use it
appropriately.

Whether you're using "Lean Startup" or traditional product development, you're
going to encounter people who say "I want feature X". Rather than simply
taking this as a request for a feature, you should act as if they said "I am
trying to do something with your software, but I can't, and I think that if I
had feature X I could do it."

Figuring out what that something that the customer is trying to do is
invaluable. Then you need to figure out if you're going to help them do that
something or not, and how to do it.

~~~
adii
Good point and you'll see that the title of my article is more provocative
compared to the article itself, where I advocate exactly what you've said.

The most important point I tried to make was that generally one should avoid
feedback this early, but if you decide to get feedback, be cautious in how you
use it.

------
ashmaurya
It's not the customer's job to know what they want. \- Steve Jobs

Customers are great at articulating problems but whenever they start
suggesting solutions, I try and get to the root problems whether it's before
Version 1.0 or a feature request after.

Before launch I do talk to customers - the first is an interview, second is
for feedback. The interview is to learn about problems and how they solve them
today (existing alternatives).

Armed with that knowledge, I then build a "demo" (prototype, mockup, etc.)
that I show to them once more to validate this "will" solve the problem before
I build the real product/feature.

For a more detailed write-up: <http://www.ashmaurya.com/2011/07/how-we-build-
features/>

\- Ash

------
bromley
Speaking to potential customers about the specifics of feature implementations
might sometimes be a bad idea before version 1.0, but I definitely think it's
good to speak to potential customers. It's by far the best way to understand
your market - figure out what problems they face, what they (think they) want,
and what they might be willing to pay for it.

I've spent the last year working on a new product. (What can I say, I'm slow!)
At several points I've thought that I've spoken to enough potential customers,
that I've got all the information and understanding of the market that I need
to go on with... And then another potential customer pops up, I speak to them,
and gain valuable insights that I never would have imagined I was lacking.

These conversations haven't really slowed me down; if anything it's the
opposite. They've helped me make decisions that I was struggling with and
they've helped me to prioritize what's important. I think they've helped me to
avoid costly mistakes. Every month I feel more comfortable with the product
that I'm soon to launch.

And I can still change direction after launch, if it makes sense to do so.

It's not just about figuring out what features your product should have. It's
about figuring out what types of people (or companies) are going to want to
buy it, which you can most profitably target, how much they're likely to want
to pay, and how best to sell it to them (e.g. what about your proposed product
really pushes their buttons?). Understanding this stuff affects so much, like
your choice of product name, how you should market it, and price it. Figuring
out the initial feature set is just part of it.

------
toddmorey
Guy Kawasaki has a great story here. As he tells it, customers surveyed about
the Apple II said they wanted more memory, more color, and more expandability.
What Apple delivered with the Mac was less memory, no color, and no
expandability. The point is that customer feedback is great for gathering
ideas to evolve a product, but you'll never get customer surveys to suggest an
outright revolution.

~~~
adii
I've never heard that story before, but it sounds just about spot-on. Do you
have a link to it perhaps?

~~~
toddmorey
I believe it's in his book Rules For Revolutionaries. I remember it as audio
of a speech he gave. I'll try to find that speech.

------
iamgoat
Good reasons. Having a channel for customer feedback doesn't hurt, but
ultimately it's up to you and what you feel is good for your product/business.

You may have a list of 10-15 projected features based on the direction you see
the product going. However, once your customers begin using your product,
their ideas may drastically change your product. For better or worse.

That's all part of the fun of running a business. There will be days where
you'll be on top of your game and feel all your ideas will make your business
a success. Other times you'll feel everything you know is wrong. The proof is
in the pudding and you get to choose the ingredients.

------
rglover
Adii definitely makes a lot of good points here. Recently, I've been speaking
indirectly to people that are in the industry my product relates to. While the
discussions are exciting, I've noticed myself trying to creep more and more
features into version 1. It's a bit of a distraction and requires a conscious
effort on my behalf to prevent it (i.e. more work). That being said, I do feel
there is some value in speaking with customers before v1. Knowing whether or
not there's a demand for your product and learning what's really _wrong_ with
what's available is a great thing. It's up to the entrepreneur, though, to
make sure that those ideas don't cloud the focus of your current work, or make
you feel as though they have to appear in v1. Take notes, not action (just
yet).

------
fredBuddemeyer
totally disagree. our site littlebiggy.org is in open alpha though not
promoted. search has brought us a half a million visitors so far. we dont ask
them what features they want so much as see how they use the site and respond
to their criticism and questions.

had we not done this we would certainly be coding a myopic vision. user
feedback has changed our functionality about 75% and much of what we do now is
based on real engagement numbers.

btw this happened by accident. one of us left a dev sandbox open to crawling
and people started contacting us about the site before we even realized it.
from that moment forward everything has improved and a lot of wasted
development time has been prevented.

~~~
ThomPete
That's not really Customer feedback. It's analysis and yes that is very
valuable, but different than OPs point

------
mattraibert
tl;dr ask short targeted questions that are designed to test a specific
hypothesis.

Working on a very young lean startup and as we've gotten off the ground we've
struggled with these problems. We'd ask what they were looking for and we
heard about big integrated solutions and lots of little (low value?) features.
It didn't seem like this was leading us in a lean direction at all.

What we ended up finding much more valuable was to identify specific
hypotheses about the problem we were trying to solve. And then set up a
specific "experiment" to test our hypothesis.

An example would be helpful: our original idea was that we would make a system
for collecting conference session evaluations -- replace the paper eval forms
with a web/smartphone based version. We identified conference organizers as
our market. Thus our first hypothesis to test was, conference organizers care
about collecting session feedback.

We called and emailed a bunch of conference organizers with a really quick
questionaire: do you collect feedback? how? what do you do with it? how much
do you spend on it?

That's it. At the end of 10 responses, it was totally clear that we were
barking up the wrong tree. Pivot.

------
nhangen
I agree with this, and it's the same approach my partner and I have adopted.
It's hard enough to build and focus on a product with internal problems and
suggestions, much less adding customer/tester feedback to the mix.

We have a vision, so we build it and get feedback along the way. Sure,
sometimes people have great ideas or new ways to do things, but if you start
taking feedback too soon, it becomes the customer's product and not your own.

------
perfunctory
Avoid using that terrible tiny font for online articles.

------
rvkennedy
Feedback is not advice. This article is saying "don't ask your customers for
advice before 1.0". I'd say, " _never_ ask your customers for advice". But get
feedback - preferably from a small sample size that you can talk to in-depth
about their experiences. But you're looking at what they're experiencing and
trying to learn your own lessons. You are _not_ asking them what they _want_.

------
radley
Bzzzt. Nope, sorry, can't agree =)

The correct answer is "be comfortable and courteous receiving redundant
criticisms on incomplete features."

------
jsavimbi
I believe that's what a Beta used to be before the 2.0's started tagging
everything with it and making incremental changes to their work based on
incessant user feedback. The problem today is that an entire plethora of
applications target the many pieces required to solve your problem if not the
core problem itself, so you either have to include a baseline of functionality
expected in todays's app and risk being flushed away or you actually
revolutionize the space from the ground up at the risk of total failure. It's
a choice you have to make and with the speed of today's releases and the
reactions of social media (see: Color) you need to strike a balance between
vision and feedback instead of focusing on point releases, which are in
themselves kind of pointless.

tl;dr: this article addresses software as it was developed ten years ago and
includes a most horrid blog theme to boot.

