

The problem with a Lean Startup: the Minimum Viable Product. - vital101
http://paulkortman.com/2012/11/21/the-problem-with-a-lean-startup-the-minimum-viable-product/

======
antoncohen
An MVP is not about building a bad product, it's about testing your biggest
assumptions and leaps of faith.

In the case of ThingShare, what are your big assumptions that are core to your
business working? I would say it's not whether people will list their games
online, that's trivial and not really required by the product. It's not
whether people want to rent games, that's proven.

I think your two biggest leaps of faith are (a), that people are willing to
lend their expensive games to a stranger for a small sum, and (b), that people
will be reliable about sending their games to other if your product works by
mail or if it works locally, that they will be willing to meet a stranger in
person to exchange.

To test those assumptions you don't build a broken website and automation
system. You have a web form, using Wufoo or Google Docs. Have people list the
games they want to rent and the games they want to lend. Then you search the
database/spreadsheet, make matches, and make all the arrangements. Make pre-
paid mailing labels, give them directions to where they are going. Do
everything, just do it all manually, you only have 30 customers.

In the match.com example, you don't build the next match.com. That is, you
don't build a whole site with a bad UI/UX. By saying you want to build the
next match.com you are really saying that match.com is doing something wrong,
and you can do better. What is it you can do better? What is your assumption?

I don't know match.com, so I'm making this up. Maybe your big assumption is
that the reason match.com doesn't work is because it doesn't take into account
financial parity. So you offer a service that will match people up with others
that make the same amount of money. No algorithms, you may not even need a
website, just match people up. If that works, if people go for it, if they
like their dates, test your next big leap of faith (maybe that people won't
lie about the income?).

~~~
namtrok
So once you test those assumptions via elbow grease, do you build out the full
product then?

No, Lean Startup focuses more on multiple iterations, aka MVPs. Test the next
feature, the next function. This is what ThingShare has been doing all along,
while still completing shares.

What people have assumed is that we didn't complete the process, where the
website failed to complete the task we took over manually. It worked well, but
users were still disappointed by the site (not by us).

Oh, and we have significantly more than 30 users, that was a point in time
with a very early iteration.

~~~
antoncohen
Yes, Learn Startup focuses heavily on testing so as not to waste resources on
a product or feature no one wants. But that is not what an MVP is. The MVP is
designed to test _leaps of faith_. You learn from the MVP, iterate, test
again, continue the build-measure-learn loop.

"Every business plan begins with a set of assumptions. [...] Because those
assumptions haven't been proved to be true (they are assumptions, after all)
and in fact are often erroneous, the goal of a startup's early efforts should
be to test them as quickly as possible." - The Lean Startup, page 81

"Unlike a prototype or concept test, an MVP is designed not just to answer
product design or technical questions. Its goal is to test fundamental
business hypotheses." - The Lean Startup, page 93

------
MattRogish
"Thus if in those days you tried renting a game you’d receive an email saying
“Congrats! your rental of Assassin’s Creed 3 has been approved! Connect with
the lender via Facebook to work out the details.” User? Meet cliff, here let
me help… _shove_ "

Umm, a big part of the "MVP" is don't automate when you can use manual labor
to test out a hypothesis. That doesn't mean your _user_ has to do the manual
labor. Why didn't you reach out to all of your lenders ahead of time and prep
them for the relationship? You certainly had few enough that you could do it
by hand. If you saw a bunch of folks contacting you wanting to rent it, you
could make the introduction and let them take over. Or you could hand hold the
entire way.

In short, just because it's "smoke and mirrors" doesn't mean it can't be
functional. It's just that _you_ take the place of working code! When you get
enough users to prove the hypothesis (and reach scaling problems of doing it
manually) only _then_ do you write code to automate it.

Even more importantly than that is you now have a relationship with your
super-early adopters. You can talk with them, do user research, UX testing,
etc. That's something you miss out in this really bad implementation of an
MVP.

~~~
namtrok
Fair critique. I didn't write about the time we spent contacting our users and
enabling the shares to happen. The key is that they expected the site to
automate the process, and were always disappointed when they received a manual
email from us. We have the relationship, but it's one that starts with an
apology instead of a "How can I help you?"

~~~
TillE
I don't understand. As pointed out, the whole idea is that you're doing
everything that code would do, maybe even better. The only possible
disadvantage is the speed of handling requests. Why would customers be
disappointed with that?

~~~
namtrok
I cannot defend the reaction of users, they just often expressed
disappointment with a "broken" system.

Like going to an ATM and then getting a phone call saying, "Hi! I'm your bank
teller processing the check you just deposited into our ATM" It's a strange
experience and they expressed disappointment while referring to it as broken.

~~~
richardjordan
I think this aspect is often overlooked in the MVP discussion. Customers are
trained to have certain expectations fulfilled and when they're not met they
perceive things as broken, as you observe.

I think the examples of MVPs that have worked best appear to be targeting
other startups or folks in the industry with a high degree of tolerance for
failure and understanding of how this all works (and therefore patience with
the process).

But most real-world customers end up disappointed with this kind of thing. So
far as I can tell a lot of MVP-driven startups just accept that they'll lose a
percentage of their early adopters over this.

Worse, and I have some experience of this with a startup of my own, often MVPs
come with the need for investment $$ to make them truly viable. So what to do?
Well get 10,000+ folks signing up and using your system, in just a few weeks,
and showing it's got great demand (that was a big number for the space). Yet
the more you ramp up early adopters the bigger the cliff when you still hear
crickets on the funding trail and never manage to afford to give those
customers - many of whom have invested a lot of time and energy in your
product - what you promised them.

Many folks are okay with this. There's something almost sociopathic about many
young inexperienced entrepreneurs who don't appear to think about the real
world consequences of abandoning their early adopters (and their efforts and
data).

~~~
wtvanhest
What's your company?

~~~
richardjordan
Current company or the one I'm referring to when in the post above (which was
prior startup)?

------
peteforde
I have noticed that startup ideas advanced by the type of folks that might
read HN are already pretty minimalist compared to the pre-Lean Startup era.
That's not to say that there aren't glaring exceptions and abuses of perfectly
good investment capital, just that our dials are set to "what are the blocking
features required, and that's it". For me that started with reading the
37signals blog; HN/MVP/4 Steps came later.

On the other hand, I assure you that non-technical founders that approach us
for development help are not on the MVP train yet in 2012. They want it all,
on desktop and every mobile platform. Helping them scale down is one of the
most valuable things we can do for them.

The takeaway for me is that I find myself largely agreeing with this article.
I've been involved with projects that released too soon and it wasn't that the
early adopters were forgiving or not. No, they just came once and were
silently let down by what we couldn't yet quite do and never returned. Those
early members were not ransoms but carefully chosen for their background and
interest in our domain.

It turns out that when you blow your first impression with the folks you need
for your Tipping Point... it's hard to move the needle after that.

The answer is obvious: an Eric Ries vs Malcolm Gladwell cage match.

~~~
namtrok
I'd love to see that cage match ;)

Instead of inviting those who would help you achieve your tipping point to an
MVP do you need to invite the non influential people? At least until you have
your feature set in place?

~~~
richardjordan
It still requires a willingness to burn early adopters - those who put most
faith and trust in you - in the cause of learning.

It's not that I advocate against MVP/Lean Startup - I've been following the
movement since it was merely a set of unorganized blog posts. But like all
things it's not to be followed blindly and uncritically in a one-size-fits-all
manner. There can be real consequences of burning your early users just to
prove/disprove theories about your product.

------
tzaman
You're focusing on the wrong term with your thoughts. It should be 'minimum'
part that is in question here. If you're building a new GPS service and need
actual satellite development and launch, then minimum may be millions of $.

MVP is not really a product, it's a customer discovery process. If you have
that in mind, it becomes much clearer.

We spent over 3 months in development plus 10k in expenses for
<http://beta.carmivore.com> and we're currently in beta testing. Luckily, we
went for minimum and early adopters already gave some valuable feedback.

~~~
stevoyoung
I agree with this. I think a better way of thinking about MVP is by calling
your V1 a "minimal compelling product". Your first version really needs to be
minimally compelling so you get good usage which leads to good feedback, like
you did. It's really a hard thing to decide but it's very important. Go to
minimal and you will fail prematurely - too much and you will never launch.
More thoughts here: [http://www.stevoyoung.com/post/32933259890/forget-your-
mvp-b...](http://www.stevoyoung.com/post/32933259890/forget-your-mvp-build-a-
mcp)

~~~
wpietri
The nice thing about failing too small is that you have plenty more chances to
get it right. The trick is to do your small failures with small audiences and
with minimal brand impact.

------
mbesto
Ugh. No business building methodology will _ever_ be scientific enough to be
reproducible in every situation. Stop pretending like it will.

When preaching an MVP to my clients I make it very clear that the point of the
MVP is not to build a successful business but to validate what problem your
product is trying solve. This is very hard for many people to stomach, because
everyone thinks their idea is special. Nor does anyone want to put money into
an idea that they don't think will work. This happens in IT and this happens
in startups...ALL. THE. TIME.

------
jaredcwhite
Good thoughts on the limitations of the Lean Startup methodology. I actually
like the concept of the MVP, but I tend to think that a lot of companies' MVPs
who are trying to follow the LS method aren't really viable.

The whole "put up a landing page and count signups" idea just turns me off
completely. However, if you can actually build a usable product that only does
one thing really well and avoids excess in every other way, then you really
have something to test. But wait! Why spend all that time and money if you
don't know it's going to work? Exactly. Welcome to the world of being an
entrepreneur. There aren't any magic bullets here.

I spent 13 months on a bootstrapped, sideproject startup and clocked over 500+
hours. I'm now testing how to market the product and start growing revenue.
What, am I crazy? No, I simply refused to release a product before it was a
product. Am I shafted if it doesn't take off? No, because what I learned in
building it (not to mention I'm using it myself and I love it) was worth the
entire exercise.

~~~
hndude
If you don't mind me asking, what is your side project startup about? I'm in a
similar boat, working on some small things that I use myself and I'm
considering trying to build into products for the public. Why don't you find
people that so desperately need what you have that they are willing to use a
half-finished, unpolished version of the software/website?

------
badclient
Great post. Lean Method likes to portray that it is merely looking for some
essential truth. However, I have seen multiple times now where two teams will
try to follow lean to validate an idea and one will have it validated while
another will fail.

In addition, Lean puts emphasis on isolating variables but I don't see enough
about the use of judgement. Instead I read how we should pursue it like a
scientific experiment. But what does that mean? In a scientific experiment you
are anal about every little variable and constant. Last time I tried to do
that while practicing Lean it ended up being the opposite of Lean.

This is more a criticism of how Lean is positioned to new people than the
method itself. Most studies I see are _not_ people who are following the
specific Lean philosophy but kind of naturally developed their own. Most
failures I see are guys finding it difficult to employ judgement after reading
Eric's book. I think he can do a better job of explaining nuances and
exercising judgement when doing Lean.

~~~
hndude
Care to share the idea that was validated by one team and failed with another?

~~~
mertd
That is the problem. It can be the same exact product just a different
randomly selected subset of people (or even same people who slept better last
night) and you could get wildly different feedback. Reading the book, I
thought there was too much faith being put on instantaneous observations,
which is are very noisy signals just due to the fact that the world is random.

------
debacle
Maybe focus more on the viable and less on the minimum? Seems a bit long
winded when the tl;dr is "We had a few bugs that turned out to be kind of show
stoppers so MVPs suck."

The real meaning behind "MVP" is to redefine done as "An executable critical
path, and whatever else required for that critical path."

If you build a bridge without handrails, people will still use it if it means
they can cross that terrifying chasm that was bothering them before.

If a few planks are using, will they still use it? Some of them will, some of
them wont.

If one of the beams is missing, will they still use it? Likely not.

The bridge is different for every company, every userbase, and every industry.
But if you're trying to sell people on the bridge, you should be an expert on
your bridge. It sounds like the guys at ThingShare weren't.

------
mgl
I start to believe that the real trick is not to identify MVP as actual
software of minimal scope but really to try and emulate your business by
running it manually. Unless you are building a strict application (like a
mobile game or another CRM app) but rather providing a software-backed service
just reach out to potential users and try to sell and deliver. Not only you
won't have to waste any time on building any speculative software solution but
you should also learn a lot talking directly to your users.

The false picture we frequently have is that the thing we sell and should
focus on is software, whereas it's really the service and experience, just
automated by a software component.

------
mlader
I'm a little put off by the author's reaction to users suggesting features
that didn't make the MVP cut. It's great that your users are requesting
features that are already on your roadmap! That means you're potentially on
the right track or at least know a bit more about your users.

When you complete them, you can tell your users that, "we listened to YOU the
community and have delivered what you have asked for". You'll be able to keep
those early users through the early stage roller coaster much more easily.

Your job isn't to be smarter or always one step ahead of your customers--it's
to provide value to them so they can't live without you. Otherwise, sounds
about right to me =)

~~~
svachalek
Exactly -- (1) you are getting direct feedback, which shows interest and
engagement (2) that feedback is pointing in the direction you were already
going; shortest path, meet straight line. The main thing is not to put TOO
much stock in these requests as people will often ask for many things they
wouldn't actually pay for.

------
polskibus
Disclaimer: I have not read the lean-startup book. I have read the summary of
lean-startup methodology.

Considering from my experience with lean methods in general (think Toyota
Way), I'd like to share what I think is the best book on Lean in software I
have encountered so far: [http://www.amazon.co.uk/Lean-Software-Strategies-
Techniques-...](http://www.amazon.co.uk/Lean-Software-Strategies-Techniques-
ebook/dp/B004A16KGI/ref=sr_1_1?ie=UTF8&qid=1353570421&sr=8-1).

There are many lessons to be learned from other industries, startups can also
learn from projects done in big corporations. There are techniques to obtain
what Paul describes as "MVP" - for example Analytic Hierarchy Process. It is a
great tool for prioritizing values and requirements coming from many, often
conflicting sources. By the way, the book I mention shows practical examples
on how to use it in software. What I am a bit worried about is that the lean-
startup may be hiding the Lean complexities by forgetting about some of the
Lean principles (value, value stream, flow, pull, perfection). They are all
very important, choosing one of them (value) over others (flow for instance)
is not going to create a truly lean company in the long run.

------
pseut
This quotation seemed strange: "Then it gets even richer: The users suggest
features which already are on our roadmap but were cut due to the word
minimum."

I thought that was the whole point of the MVP. Your initial users just told
you which features you should add next and (by implication) which of the
others on your roadmap you should wait on.

Plus, it sounds like you also learned you need to iron out that Facebook spam-
filter issue, which I'm going to assume was lower on your priority list pre-
MVP than after, and which you are probably attacking with a little more
urgency now that your product's out there than you would have otherwise.

------
mediascreen
To me MVP is about getting to the core of what's different about your product
and then find a way to test that without implementing anything else.

It also helps you avoid vague ideas like "build Facebook - but make it
better".

~~~
namtrok
but if what's different about your product is an algorithm change on match.com
you still need everything else match.com has to prove the benefit of your
algorithm

~~~
wpietri
If what's different about your product is a minor tweak on an existing giant,
then you have bigger problems.

Even if you are trying to beat an incumbent, you probably don't need
everything they have. If there is a true unmet need in some audience, then
they will be willing to put up with quite a number of missing features because
you are solving their problem better than the incumbent.

Or if you're really just trying to test the hypothesis, you can cheat. Proxy
all of match.com and just insert your own rankings and logo. Or just sell it
honestly as something that helps you find the best people on match.com. If
people buy it, then you know you've got something. If not, then you have
learned that nobody cares enough for you to go build match.com.

------
yo-mf
What they do not tell you in lean startup; your early users are your unwitting
product testers. Thus the calculus becomes how quickly you can plug the gaps
and deliver the real MVP before your users bail en masse.

~~~
richardjordan
It also assumes you have no social capital that can be burned by channeling
folks towards trying something on your word and then finding it isn't - in the
eyes of the user - a finished product, it's broken or under-featured.
Reputations can be ruined like this.

It's not all Dropbox posting a signup form with no product behind it or Steve
Jobs building the iPhone from whole cloth in secret with nothing in between.
There's a continuum and startups can find their own spot on that path. IMHO

------
mbh
When I read the book and this post, also reading almost all the comments here,
I feel like one can just build a survey with direct questions about the
assumptions and beliefs and get answers. Why even elbow grease? or do anything
else?

Is it that users need to be in the "situation" to really give you correct data
driven instinctive answer? While if you ask them a set of questions, the
answers might not be correct or they might not answer?

~~~
wpietri
You've got it right. People say a lot of things that aren't true about
themselves. Especially when you're asking them hypotheticals. Surveys have
their uses, but they are not proof.

The only solid evidence that you have a real business is that you get money
from people for something you did for them. So you get that evidence with the
smallest experiment possible. That's the MVP.

------
sopooneo
I take a issue with a lot of the points in this article but agree pretty
strongly that Ries should have picked a different name than "lean". If you
continually have to clarify that you don't mean something else (cheap in this
case) then you name is not good. No matter how much you can justify that your
audience is being dumb in their interpretation, it is still on you to use a
word that gets the idea across without confusion.

~~~
wpietri
This is a problem with every name that conveys actual meaning. If you're
trying to sum up something complex in 1 or 2 words, people will inevitably
have misunderstandings if all they go on is the word.

He picked the name Lean because a substantial part of the philosophy comes out
of Lean Manufacturing. And because an essential part of it is striving to be
lean. Works for me.

~~~
sopooneo
Yes, but it is a problem that is true to _different extents_ depending on the
word. The origins of Ries's philosophy may justify his use of "lean", but if
that was not the case, I think it would have been better and possible to find
a word with a higher meaning to confusion ratio.

~~~
wpietri
For example? Note that it has to have equal or better marketing potential.

------
saryant
I love this post. I'm in a similar situation in trying to figure out what the
MVP for our upcoming launch needs to comprise.

We've done some market validation but aren't totally convinced it's there yet.
At this point, our only recourse is to build _something_ , put it in the wild
and see if anyone signs up.

------
paulsutter
Your initial users may have been inconvenienced, but they weren't harmed.
Relax - you don't need to worry so much.

Seems like you really found the right balance between minimum and viable.
Great work.

EDIT: rewrite to be much shorter

~~~
hndude
You sound like a very Zen (if you don't like that term, calm) person :)

------
erik_p
I think the article is confusing software building with product testing. I
think the intent of the lean startup MVP concept is to enable learning about
your product's market fit as quickly as possible.

