
Ask HN: Survivor Bias. Why don't we focus more on startups that fail? - econner
I think this comes from a book somewhere or other but it was paraphrased to me:<p>During WWII planes that made it back from combat missions were analyzed to be reinforced.  They found that these planes were shot in the wings and tail so they reinforced those areas.  Really though, planes shot in the fuselage were much less likely to make it back so they should have reinforced that area.  They were biased by trying to avoid the problems the survivors faced.<p>So my question is: why don't we try to learn more from startups that fail instead of focusing so much on the successes?
======
antiterra
The National WWII Museum implies that your WWII anecdote is the inverse of
what actually happened. That is, that returning planes were shot in the
fuselage, and a WWII statistician recognized survivorship bias in proposed
reinforcement and concluded that the areas not hit on survivors should be
reinforced.

See: [http://www.nww2m.com/2012/11/scitech-tuesday-abraham-wald-
se...](http://www.nww2m.com/2012/11/scitech-tuesday-abraham-wald-seeing-the-
unseen/)

<http://en.wikipedia.org/wiki/Abraham_Wald>

~~~
econner
Ah. Smart statistician. Thanks for that.

------
pg
We pay a lot of attention to the causes of failure. Much of what I've written
about startups is about pitfalls and mistakes. I don't name names, but I don't
think you need to.

~~~
blargh123
hi pg.

i don't know you, this website, or anything about anything, but i felt a duty
to inform a fellow internet user that someone has commissioned others to
compile your comments for this site. here's the link:
[https://www.elance.com/j/scrape-gather-all-comments-
person-o...](https://www.elance.com/j/scrape-gather-all-comments-person-on-
discussion-
board/35373787/?backurl=aHR0cHM6Ly93d3cuZWxhbmNlLmNvbS9yL2pvYnMvY2F0LWFkbWluLXN1cHBvcnQ=)

it could be totally fine, or creepy, or it could just be you wanting your
stuff compiled, but i felt it my duty to tell a fellow internet user that
someone wanted their stuff compiled.

if i'm breaking any sort of rules, again, sorry, i don't know anything about
this site or how to navigate it; i also obviously don't know how to pm.

cheers.

~~~
pg
[http://www.hnsearch.com/search#request/comments&q=by%3Ap...](http://www.hnsearch.com/search#request/comments&q=by%3Apg&sortby=create_ts+desc)

------
IceCreamYou
There are an infinite number of ways to fail and only a few ways to succeed.
If you learn from failure, you have learned one way that doesn't work out of
an infinite number. If you learn from success, you have learned one way that
does work out of a small finite number.

Obviously there are some ways to fail that are much more common than others,
but those tend to be based on lack of action, e.g. failure to actually ask
users if they want to use your product before building it. There is plenty of
material on those common sorts of failures, it's just usually phrased
constructively, i.e. "how to do SEO" rather than "we died because we posted
duplicate content on every page of our site."

~~~
baddox
The hard part is pinning down the _cause_ of a successful startup. Most people
just point at highly visible things, and make a claim like "StartupX is
successful because the founders worked extremely hard, the office culture was
well-developed, and there were lots of team building activities." The problem
is that this ignores the 5,000 other startups that did all those same things,
but failed. Perhaps it turns out that StartupX really succeeded because they
had a sales guy with lots of good connections, etc.

~~~
sek
You have a good point here.

More prevalent in the financial industry, you have a lot of successful
retirees who hold talks and write books what kind of tea they drank.
Statistically nobody beats the market, so these people had very likely just
luck.

Causation is not relation, everybody knows that. But it is damn hard to figure
out what and often the people themselves are biased. I would say a big
percentage of tips you get this way are totally useless, but to figure out
which one of them depends on your judgement.

------
pbiggar
I wrote up my failed YC startup here: [http://blog.paulbiggar.com/archive/why-
we-shut-newstilt-down...](http://blog.paulbiggar.com/archive/why-we-shut-
newstilt-down/)

It was definitely a good move. More people read that blog post than heard
about my startup in the first place, it got covered in loads of tech press,
and people still ask me about it 2 years later. I get an email every few
months thanking me for writing it, and I'm asked to do the occasional
interview about it too.

It also allowed me to introspect a lot of what went wrong, and what I could
have done better. One of the major reasons my current startup
(<https://circleci.com>) is doing well is what I learned from the
deconstruction.

~~~
thomson
Just wanted to echo common sentiments that your blog post was a very well-
written and yet personal treatise on how and why startups fail.

Another good pair of posts comes from Chad Etzel (previously Notifo):
<http://blog.jazzychad.net/2011/05/02/startups-are-hard.html>

[http://blog.jazzychad.net/2011/09/08/fouling-out-moving-
on.h...](http://blog.jazzychad.net/2011/09/08/fouling-out-moving-on.html)

------
edanm
When I joined the startup scene 3 years ago, I mostly saw success stories. Now
that I've been through my own failed startup, I see many failure stories.

I believe that two things have changed:

1\. The field is maturing, so more people are talking about failures, the
downsides of entrepreneurship, etc.

2\. I'm growing as a business person, so I'm reading a larger variety of
things, not just stories that reinforce my beliefs.

So it's possible that, like me, you're so focused on reading positive startup-
reinforcing articles, that you're missing out on some of the articles which
focus more on the failures and things to avoid.

Oh, and definitely read pg's articles - they have a lot of truth in them,
compiled from one smart person who's seen many startups both fail and succeed,
and who is specifically looking for the reasons and patterns behind it all.

------
andrewcross
I think there's two major components to this:

1) Writing about your failures sucks. My first startup got to 6 figure revenue
within 18 months, but we really struggled to grow past that. I could write a
full debrief on why we failed, but it's pretty low on my "things I want to do"
list. I'd imagine quite a few founders with failed startups would have a
similar viewpoint.

2) A lot of failures just aren't that exciting. I'd venture a guess that most
startups fail because they gradually lose their passion for the problem. When
you aren't growing for an extended time, it wears on you. A lot. The only way
you survive is if you're obsessed with the problem you're solving. Those that
aren't would just slowly fade out.

------
lsiebert
I'd note that this bias extends beyond startups; see
[http://jeps.efpsa.org/blog/2012/06/01/falsification-of-
previ...](http://jeps.efpsa.org/blog/2012/06/01/falsification-of-previous-
results/)

studies of failures don't just reveal things that increase failure rates, they
prevent false positives.

Imagine X is some innovation that is being talked up because recently several
companies had or used X to succeed.

If 19 out of 20 times X makes a startup more likely to fail or if X has no
meaningful effect on failure rates at all, or if X only helps in situation Y,
but you only hear about the times that people succeeded with X, you might try
to use X, to no benefit or to actual harm to your business.

If startup success is rare, then trying to generalize only from instances of
it may lead to errors. We have more instances of failures, therefore we can
learn a great deal about commonalities in startups that fail. Of course the
smart thing to do is to make a hypothesis about X, and compare startups with X
that failed and succeeded to startups without X that failed and succeeded to
see if there is a significant effect on the rate.

I'd note that errors of judgment about what causes startups to do well or fail
get fed into the investor system. If X is useless or harmful, but your
investors think it's a good idea, you will have problems.

------
plowguy
I think the answer is pretty simple: failed startps are insular. Founders who
are able to discuss them won't, either because they've moved on, or because
it's painful, embarrassing etc. Discussing failures would be more useful, I
imagine, than reading about successes. If such a startup graveyard existed,
I'd probably visit quite often.

------
pyre
\- The failure may be a sore-spot for those involved, so they might not want
to do a postmortem.

\- Unless insiders are willing to speak up, there may not be too much to be
said from the outside.

\- A lot of start-ups that fail are uninteresting. What can be said about the
tons of TODO web apps that have probably been created and failed.

\- Those involved may be reluctant to speak out because they view the lessons-
learned as their leg up on the competition when they get into their next
start-up.

I agree that there are a lot of interesting things to discuss regarding failed
start-ups, though.

------
spartango
I think on a person-to-person basis there's a lot of talk about startups and
companies that have failed. People who have worked with startups that ended up
going under have no discomfort talking about the causes and errors, which
subsequent companies learn from. I think the experiences and lessons are
passed on fairly well, it's just that this exchange happens out of the public
spotlight.

------
jacques_chester
For those interested in a scholarly study of common failure modes for various
fashionable Grand Business Strategies of the 80s and 90s, read _Billion Dollar
Lessons_ by Carroll and Mui: [http://www.amazon.com/Billion-Dollar-Lessons-
Inexcusable-Bus...](http://www.amazon.com/Billion-Dollar-Lessons-Inexcusable-
Business/dp/1591842891)

------
dbyrd
Good point. I was thinking that a lot of the characteristics of great
entrepreneurs are really just things that make them more volatile. So for a %
of the people they will be more successful, but for another percentage of the
people with the same characteristics they will be a lot less successful.

I have no evidence to back this up, just food for thought.

------
shawn-butler
I think another reason is that no one likes to acknowledge the uncomfortable
truth that a significant factor in the success or failure of tech startups is
pure chance. It's the same reason so much of recorded history is false. It
seems to be hardwired into human experience to prefer a good narrative over
truth.

------
Zev
Once you've failed (and maybe, given your post-mortem), what else is there to
do? You move on to working on something else.

To extend your analogy: during WW2, they didn't study the _same_ plane every
time a plane was shot down. They studied a the one that was hit, and moved on
to the next one after that.

~~~
dfc
Your missing the point of the analogy. The planes that should have been
studied were the same planes that never RTB'ed so they could not be studied.

------
daa
There is a notable exception: <http://thefailcon.com/> "FailCon is a one-day
conference for technology entrepreneurs, investors, developers and designers
to study their own and others' failures and prepare for success."

------
smirksirlot
Availability heuristic? Successful startups survive, do well and people hear
about them more (news coverage, PR, using their products etc.) It is hard to
discuss a company when you've never even heard of them.

------
RyJones
In my case, NDAs, mostly. Plus, as a savvy recruiter once told me, the Bay
Area is big, but the Valley is small.

------
jborden13
Agreed. Probably an equal number of lessons to learn from the failures as from
those still slogging it out.

------
hayksaakian
Failure is bad for SEO. Unless the failure is massive.

Many startups fail for uninteresting reasons.

~~~
TheOnly92
Well it is my opinion that failure stories are far better than successful
stories because success is difficult to replicate but failures can be easily
avoided.

------
rjzzleep
same reason we like heroes so much, even though most heroes didn't do things
on their own.

------
bravoyankee
I agree. It makes for a lot more interesting story too.

