

Business Advice Plagued by Survivor Bias - yarapavan
http://blog.asmartbear.com/blog/business-advice-plagued-by-survivor-bias.html

======
mediaman
Jason, this was a great piece and more should do themselves a service by
reading it.

It's one reason to pursue 'smaller' opportunities with lower variance of
return expectations but higher median returns -- for example many niche
businesses -- than industries with gigantic return expectations. The latter
attracts far more entrants, and while _somebody_ gets rich, the median player
does not. But this isn't clear to new entrants because of survivorship bias.

Somewhat related to this is the data availability bias. Big opportunities tend
to have much better data sets, and so are easier to study and receive more
attention. Many niche industries are closely private and are smaller, so do
not have much publicly available data, making study difficult and attracting
less attention. Therefore like moths to light bigger industries receive more
study and more entrants even though their median expected economics are worse.

~~~
mhartl
_It's one reason to pursue 'smaller' opportunities with lower variance of
return expectations but higher median returns -- for example many niche
businesses -- than industries with gigantic return expectations._

This neatly encapsulates much of my current business strategy.

------
swombat
_Do you read business blogs where the author has failed three times without
success?_

Am I the only one who regularly reads blog posts by an entrepreneur analysing
how he's failed and what he would do differently next time? They pop up on HN
quite often.

~~~
BrandonWatson
Are you talking about www.thefailingpoint.com? That's me, and I am happy to
share my experiences. They have been in the massive up and down times. Success
and failure. I find it therapeutic to write, and since my old engineering team
is in Y!Combinator this summer, I got the religion about sharing what I know.
They asked me for a compendium of all the stupid mistakes I had made so that
they wouldn't make them. =) I joked it was too much work, but it turns out
that it makes for some good essays, and I really hope to turn it into a
published work. For now, I am posting all the essays online for all to read
for free. Chapter 1 and 2 are almost fully posted. Please send feedback.

------
bigwill
Very Nassim Taleb. These are great books NT's written that discuss our
susceptibility to survivorship bias and other rational failings we have:
_Fooled by Randomness_ and _The Black Swan_.

~~~
smartbear
Yes! You're right I'm hardly the first people to talk about such things. The
Black Swan is a classic although oddly enough right now on the front page of
HN there's a critique of Black Swan that pokes holes in it. :-)

But yes it's true there's lots of good material on these subjects for those
who want to explore further. Hopefully my piece whetted your appetite -- it's
not the final word!

------
zasz
This article makes everything pg says about the reasons behind Viaweb's
success sound rather suspect, doesn't it?

~~~
mhartl
Pretty much every startup success story has an "and then we got spectacularly
lucky" part to it. In Viaweb's case, it's more like "and then, for the _n_ th
time, we almost died, then somehow didn't", but the idea is the same. It's one
of the great unmentionables in the startup world, and understandably so:
succeed and you want the credit; fail and any mention of luck sounds like sour
grapes.

I've heard a lot of startup founders talk about a lot of startups, and only
Marc Andreessen (of Netscape fame) both realized and admitted to the large
role luck nearly always plays.

~~~
mhartl
I should add that great startup success stories almost always involve founders
who were awesome as well. (Netscape and Viaweb both qualify on this count.)
It's just that being awesome isn't enough. Traf-O-Data had awesome founders,
but it failed; those same founders, Bill Gates and Paul Allen, had better luck
in their second venture.

------
yarapavan
The article quotes Peter Norvig: When a published paper proclaims
"statistically, this could only happen by chance one in twenty times," it is
quite possible that similar experiments have been performed twenty times, but
have not been published.

~~~
ggchappell
It occurred to me that this means there is a real advantage to academia's
"publish or perish" phenomenon. It means that when an experiment's results are
not what was hoped, those results are still likely to get published. The
experimenter would like to have his name on some ground-breaking results, of
course, but, failing that, he still _must_ have publications. So he publishes
anyway. Not so true in industry.

~~~
jerf
Are you sure about that? Journals don't (in general) publish negative results.
The result of the pressure to publish is not that negative results get
published; it's that the scientist has an incentive to _create_ positive
results. Not necessarily through data falsification, but through choosing
experiments very likely to generate "results", which, unfortunately, also
implies that the information content of such experiments is necessarily lower
than it could be if the scientist had more freedom to perform an experiment
that had a higher likelihood of failure.

~~~
jibiki
Googling "journal of negative results" gives a few hits. I think it's probably
possible to get negative results published, but I doubt that they look good on
the resume.

------
neilk
The takeaway here should be: if the book doesn't systematically examine the
differences between winners and losers, drop it and walk away. Sentence by
sentence, it's making you dumber.

So forget _In Search of Excellence_. Try books like _How To Be a Star at Work_
, which is based on a pretty rigorous study of employee productivity.

------
seanstickle
Talk about survivor bias:
[http://www.dailymail.co.uk/news/article-1165768/The-man-
surv...](http://www.dailymail.co.uk/news/article-1165768/The-man-survived-TWO-
nuclear-bombs-Lucky-Yamaguchi-tells-lived-Hiroshima--fled-home-Nagasaki.html)

------
ojbyrne
Jumping on books marketed at airports to harried executives isn't really
valid. There's plenty of actual business research done by professionals (and
they've heard of "survivorship bias, along with all the other threats to
validity). It's like looking at all the "For Dummies" books in your local book
store's computer section and deciding that CS research is poor.

And Harvard Business School Case Studies are meant for teaching, and their
function is analogous to science textbooks that leave out all the dead ends
and fruitless research for pedagogical reasons.

------
movix
"Planes with bullets in the cockpit or fuel tanks didn't make it home"

Such a brilliant piece of lateral thinking - thanks for sharing.

------
donaldc
This is a good point. At the same time, I believe there are actual differences
between businesses that affect their chance of success.

So the trick is to separate the 'accidental' features of successful businesses
that don't increase their chances of success from the 'essential' features
that do.

I wish the article went into how one might do this...

~~~
mediaman
One important way referenced in the article is to evaluate failures as well as
successes for the likelihood of a trait: for example Jim Collins describes
certain features that successful companies have, but does not evaluate the
pool of losers to see if they also featured the same trait.

Another example is when evaluating various corporate traits against stock
market performances, many people will choose a list of companies that exist
today and then study their history to identify correlations of various traits.
This omits those that no longer exist today, and it is a common problem among,
for example, beginning hedge fund equity analysts. Instead, one should study
the cohort from a random sample as existed at the beginning of the study, not
at the end, and then study their performances from that point forward.

------
calcnerd256
Teleology is funny.

~~~
smartbear
Did I cause you to say that? :-P

------
qu1j0t3
The winners write the histories.

------
schammy
This was a great read. I guess I have underestimated the intelligence of
bears!

