
Survivorship bias and startup hype - ifcologne
https://www.sonyaellenmann.com/2018/06/survivorship-bias-and-startup-hype.html
======
katzgrau
If you don't mind being relatively unheard of and only want to make enough
money to be independent, then it actually isn't very hard to bootstrap a
business. It does take time, effort, and a lot of mistake making, but it's
like learning anything of value. I think it makes you a stronger and more
independent person all around.

Even though this article attempts to deliver a dose of reality, it focuses on
VC backed companies which is a completely different field. VC backed startups
tend to do highly risky things in an effort to increase traction and valuation
as quickly as possible. Most notably, spend a lot of investor money in pursuit
of those things without having any real revenue. The odds of failure
skyrocket.

I've seen a lot of commenters go on about survivorship bias. I believe that
the real problem isn't being considered: the absolutely batshit advice and
business building strategy that comes out of the world of venture capital and
tech accelerators.

~~~
jasode
_> , then it actually isn't very hard to bootstrap a business._

Your "not very hard" advice also inadvertently distorts the reality of
possible outcomes. _Most_ bootstrapped businesses fail as well.

Whether an entrepreneur gets some friends with a bootstrap mindset to start a
consulting business, publish a new magazine, or open a restaurant, the odds
are that all of those will most likely fail. Any business that can provide
financial independence has long odds. (I'm not including solo businesses such
as freelance web consultant, insurance agent, home inspector/appraiser, solo
attorney, etc.)

Whether it's a bootstrap-business or VC-business, founders will still proceed
forward even if it disagrees with "rational" calculations of
successes/failures. It's human nature.

~~~
katzgrau
Yeah, but the reasons for failure are different. A startup usually fails
because the founders are convinced they can't do whatever they want to do
without investors and they never raise any money. OR, they do raise money and
eventually run out of it because they never made enough money to support the
costs that were enabled by ... the investment.

A bootstrapped business is alive as long as the founder is willing to continue
trying to make it work. So a bootstrapped business doesn't fail per se —
usually the founder fails to make money within the timeframe they'd hoped for
and gives up.

I didn't start making real money with my business until I was 4 years in.
There wasn't some magical day where I got lucky and started making money. The
first 4 years was me basically being an idiot and making every mistake
possible. It gave me the education I needed to price my product appropriately,
position it, and sell it.

Some might say I'm a "survivor" but I don't think I am. I'm just determined as
hell and didn't give up when I wasn't making any money. I worked freelance
contracts and kept the thing going because I knew at the very least I was
getting smarter even if I wasn't bringing in money, and eventually the
education would probably prove useful or pay me back. Better and cheaper than
grad school anyway, right? I sometimes think about the people who are "Masters
of Business Administration" on paper and have never actually built or run a
business.

The people I know who have successfully done it aren't geniuses with a bunch
of money and lots of connections. They all just had the same determination to
stick with it, adapt, learn, and find their way to what they want. There is
definitely some pain and self-doubt involved, but like I originally said, you
come out stronger for it.

~~~
jasode
_> A bootstrapped business is alive as long as the founder is willing to
continue trying to make it work. So a bootstrapped business doesn't fail per
se_

I'm going to rewrite your comment in more concrete terms and by doing so, you
can let me know if I have parsed your meaning correctly.

For example, in California, the annual filing fee for C-Corp or LLC business
entity is $800. So, for as long as the entrepreneur is diligently paying the
$800 each year, the business has not been formally "shut down" and therefore,
has not "failed". (Alive != Failed)

With that adminstrative setup as the baseline, we consider your next comment:

 _> didn't give up when I wasn't making any money. I worked freelance
contracts and kept the thing going [...] even if I wasn't bringing in money,_

I think this means that one can get any 9-5 day job and keep the "bootstrap
business" as a side hustle for years that never brings in significant money.
It can never be considered a "failure" because of 2 properties:

(1) the business wasn't formally wound down (just continue to pay $800/year
admin fee).

(2) a money-losing business still provides "education" and founder comes out
stronger. (Not sure if this also implies that failed VC business does not
provide education and strength)

It seems like your advice is dependent on a hyper technicality of what
"failure" means to you instead of engaging the spirit of the comparison
between VC vs bootstrapped.

I go back to your first comment which already had as a condition _" want to
make enough money to be independent"_. Many current and former bootstrapped
business founders will disagree that it's "not hard." It is extremely hard to
create a self-sustaining bootstrap business that brings financial independence
reasonably fast. (The "fast" being relative to working 30 years at a 9-5 job.)

Also, when you rely on phrases such as _" determined as hell and didn't give
up"_, you're inadvertently falling into the same _" survivorship bias"_ for
_your_ bootstrapped business which the article complains about for VC
businesses. Being "determined as hell" is an unfalsifiable attribute. Oh, you
say your bootstrapped business failed?!? It happened because you weren't
determined as hell.

I'm sympathetic to bootstrapped businesses because I sold custom software that
way but touting the advantage to non-VC businesses as one that let's you
_redefine "failure" in an idiosyncratic way which keeps the business from
meeting that criteria_ \-- is not very compelling.

In any case, if the business is in a domain that can be bootstrapped, by all
means, do that option. The founder will have more leverage. Leverage is good.

The point to my first comment is that advocates for "bootstrapped businesses"
will have the same cognitive biases as the essays for VC businesses.

~~~
katzgrau
A few points:

* When I said, "I wasn't making any money," I meant not anything of significance. I wouldn't consider a thousand or two thousand per month significant, which is where I was at for a while. If I can't live on it and hire an assistant to delegate some work to, it's not significant. If I couldn't pay admin fees it would be a failure. But yes, I did redefine failure for bootstrapped startups because I think it's valid. VC-backed startups don't have this flexibility / time to learn the hard way.

* In regard to being determined, I know this sounds like survivorship bias but you really do get better at what you're attempting to do by sticking with it. Your product gets better. Your personal network grows. You're more yourself in more uncomfortable/growth situation. Your odds of failure drop the longer you work on achieving something — not surprising.

I'd guess 99% of the people who have taken at shot at learning a second
language fail (or give up or get bored) before they become fluent. Does that
mean the 1% who made it and say "practice every day" have some level of
survivorship bias? It's kind of absurd. Bootstrapping a business to success is
like anything that takes time and effort and is based far less on good fortune
than people think.

You parsed what I said pretty deeply and I'm not that smart of guy, so if
there are flaws in what I wrote here that's understood - but you might
understand what Im trying to get at.

~~~
jstandard
>In regard to being determined, I know this sounds like survivorship bias but
you really do get better at what you're attempting to do by sticking with it

This is survivorship bias. You're a survivor and can look back on the set of
decisions you've made in the past to rationalize your current success. I've
seen countless businesses linger for years only to shutdown later and sell
their assets.

I find learning a language a poor analog here as it's much simpler and
generally within your control. More effort is more closely tied to more
results. Language learners aren't typically affected by seasonality, fads, or
market conditions.

It's all of these other factors that make analyzing business success factors
tremendously difficult. Because it's so difficult, it's easier and more
straightforward to assume "I'm successful, so my decisions were the right
ones" than "I'm successful mostly due to luck".

------
fermienrico
This is a great article on two standpoints - it provides important high-level
"abstract" advice to new founders without sounding like Sam Altman (more on
that later). Secondly, it criticizes the general human tendency to look for
examples of success, read biographies, watch interviews and think mimic those
traits in themselves thinking that they've won the ticket to silicon valley
lottery. I'd like to quote Richard Feynman, "... for nature cannot be fooled."
and the world is an insanely complex place with a number of external factors
that founders have no control over. Doesn't mean we should all give up on
starting a company, but it should be humbling to learn this and internalize
it.

Every single advice that Sam Altman gives has survivorship bias written all
over it. I have never learned anything remotely useful from his vague generic
articles and speeches. I'd rather watch Drew Hudson's startup pitch or read
about Chad Rigetti and his quantum computer venture. Or maybe learn how
Instacart managed to photograph every single grocery store item by literally
going to the store and buying everything. Sam Altman's interviews are
fantastic and I thank him for that but I think he should stop marketing
himself as a startup guru of some kind. Each situation is unique and my
personal opinion is that particular qualities such as leadership, convincing
abilities, and charisma cannot be learned. It can only surface as a result of
how one thinks and acts. This is why leadership courses are bullshit.

~~~
oceanman888
->Every single advice that Sam Altman gives has survivorship bias written all over it. I have never learned anything remotely useful from his vague generic articles and speeches.

Every and never are really strong words, I don't think he intend to market
himself as startup guru at least intentionally.

He is young and already had 10 years of experience of watching top notch
startups rises and falls. Main message I am getting from him is practically
advice about start-ups. like, do this and don't do this, then you may succeed.

~~~
onion2k
_He is young and already had 10 years of experience of watching top notch
startups rises and falls._

Is watching a few thousand baseball games enough experience to pitch at Fenway
Park?

(I mean no disrespect to Sam; I've never met him and I have no knowledge of
how good he is at advising startups. Starting and exiting Loopt shows he
definitely knows more than most of us. I'm just skeptical about whether you
can get useful experience of something passively.)

~~~
soneca
Passively? He is leading YC, not a techcrunch journalist.

~~~
newfoundglory
He hasn't been doing that for ten years.

~~~
soneca
Before that he was a startup founder, right? Then a YC partner.

~~~
sidlls
He founded a single start-up when he was 19 that failed in its mission and was
bought for a paltry sum (for YC, anyway). Other than that his experience in
life and business consists primarily of having been born into affluence and
all the privilege and advantage that brings.

It's so very Bay Area for someone like him to be president of a VC firm. His
pre-YC resume wouldn't get most people past the resume filter in almost any
other company, including a number that YC fund.

And most of his comments about business are not all that useful. They're
exactly the sort of comments I'd expect from someone blessed with great luck
and above average intellect but too little experience.

------
ksec
But the world as whole, doesn't accept Luck. Not middle or even top
management. Those who do, are actually founders, Bill Gate and Steve Jobs both
talk about how lucky they are. I mentioned Luck in most of my interview and
not a single one likes it. The older I am, the more I became aware how many of
my success or wins , are not really because I am ten times better then others,
yes it may have played a part, but there are lots of moving parts, and I am
only a small part of it. In essence, I got lucky. There is no way, a human, or
even machine could calculate what is going to happen next. The possibilities
and probabilities are endless.

~~~
bufbupa
“Luck is a matter of preparation meeting opportunity” - Oprah

By and large you can't control what opportunities sprout up around you, but
you can control how prepared you are to take advantage of them when they do.
EG: you can spend more of your free time learning new technologies, so that
you might at least be passable at them should the need arise.

I think the reason interviewers don't like talking about luck is because
they'd rather hear what you're doing to help yourself at getting lucky. How do
you best prepare yourself for the unknown opportunities coming in the future?
Sure, its only half the equation, but its the half you get to control.

~~~
6cd6beb
Generally I think you're right, but Seneca may have said that before Oprah
did.

------
tsenkov
Such a nice article. But I have to note that the linked (2013) piece is even
better: [https://youarenotsosmart.com/2013/05/23/survivorship-
bias/](https://youarenotsosmart.com/2013/05/23/survivorship-bias/)

~~~
mmt
It's also quite a bit longer, though I agree it's worth the read.

> You succumb to survivorship bias because you are innately terrible with
> statistics.

This one line is probably one of my biggest takeaways, and I wish we had a
good term for our poor-intuition-about-stats.

Recently, I've seen it conflated with "innumeracy" here on HN, but that misses
the point, as educating someone in statistics doesn't necessarily remove the
natural intuition, which remains poor. We still need to be aware of it and
the, not necessarily obvious, effects on our judgment it can have.

------
John_KZ
The massive amount of failed startups is caused by people who think this is a
fun popular way to make money out of thin air.

If you're level-headed and sensible, you can do your analysis, make a business
plan and figure out if your idea is marketable or not. Then, there's the
business-doing portion, which is extremely complex, different from your
technical work, and will make or break your company. But that only works if
you have a financially viable, novel idea. Unless you intend to scam bad
investors, you can't have a startup without real innovation, period.

The startups were meant to be a way for experts in some field to materialize
an (actually) innovative idea with economic potential, and generally an idea
that doesn't require absurd amounts of capital. This could be done in a
typical corporate environment, but when the innovation originates from a
single individual, the startup model allows him/her to better protect his
interests and grow his business faster.

When running a startup became a trend (in many areas of Europe it did, I don't
know about the Americas) there was a huge influx of people with no technical
expertise, all of whom simple wanted to be a startup owner for the sake of it,
with no plans or targets, just to be managers. These people fail over and
over, sucking up EU and local government resources with them, and dragging the
name of real startups in mud. Many of them are just the kids of wealthy
individuals with connections. They leverage these connections to get
investments and loans and waste 100s of thousands to millions on repeated
failed attempts to materialize some retarded idea, like solar freaking
highways or the 10,000th taxi hailing application on the market.

Survivorship bias is bad, but this situation would persist even without it.

~~~
mmt
> (in many areas of Europe it did, I don't know about the Americas) there was
> a huge influx of people with no technical expertise, all of whom simple
> wanted to be a startup owner for the sake of it, with no plans or targets,
> just to be managers.

I find this perspective quite fascinating, as it is something of the converse
my impression of some of the complaints I've read about the startup scene here
in the US (mostly SFBA/Silicon Valley).

Here, the complaint seems to be that (some) founders may have technical
expertise but lack the business/managerial savvy (or, again, with no plans or
targets) to turn it into a real product that makes money.

I'm unsure if either criticism, lack of business or technical acumen, is
actually fair or accurate, but it may be a subtle indication of an over-focus
on one over the other (by the investors) in each of our regions to the
detriment of overall innovation.

------
1ba9115454
To me survivorship bias is looking at someone who made it. Assigning some
charateristics to that person. i.e. Hard working, motivated etc etc.

But, forgetting that lot's of people who didn't make it also had those
charateristics.

Then you have to factor in luck. 2 Entrepreneurs with the same work ethic and
idea but one gets lucky. The difference in money generated can be huge.

------
eksemplar
I think you should stay in school. I know it’s harder t justify in America
where it’s really expensive, but the benefits can be immense if you use your
time in the educational institutions correct. It’s really the most secure path
to a meaningful life if you’re willing to work hard at bettering yourself.

That being said I think the article stumbles a little. Where there 1000 Steve
Jobs and Wozes working on a personal computer at the time? Where there 1000
people working on an global online payment system when Musk was building
PayPal?

I think timing is important, but I don’t think luck makes that much of a
difference. You don’t obtain connections by being lucky, for instance, you
obtain them because you want them and work hard at it.

~~~
blackflame7000
"You don’t obtain connections by being lucky, for instance, you obtain them
because you want them and work hard at it."\- I have to disagree. You have
little control who comes into your life outside your 1st and maybe 2nd degree
of separation. You have almost no control over who will be assigned your
freshman dormmate, your instructors, or even your boss. You can't work hard to
be put in the same dorm as mark zuckerberg. That is just plain flat out luck.
However, smart people are attracted to other smart people and that is why
smart people tend to "Create" their own luck. They dont posses any special
ability, per say, other than a desirable trait that others like them want to
exalt.

~~~
TAForObvReasons
The luck really starts with your parents and grandparents, as their financial
status generally determines your early social network.
[http://philip.greenspun.com/bg/](http://philip.greenspun.com/bg/) discussed
some of Gates' early advantages, like having a trust fund to fall back on if
risks don't pan out.

~~~
blackflame7000
But that excludes countless counterexamples with people being born with
nothing and becoming self-made millionaires. Again that's not luck its a
product of work.

~~~
TheOtherHobbes
And that excludes even more countless examples of those who work hard and
fail.

There are vastly more of the latter than the former.

Work - if it’s the right kind of work - can only ever increase your odds. It
can never guarantee success.

What’s needed is hard data about the relative influence of market fit, hours
worked, personality, network and wealth of origin, and so on.

~~~
BkBlue
Analysis paralysis is arguably as dangerous as survivorship bias.

Of course winning the genetic lottery bestows tremendous advantage. And of
course luck is a huge variable.

But the intersection of luck, preparation, and willingness is serendipity. And
every successful entrepreneur has that in spades in the end, regardless of
where they started.

Sure that sounds like another empty platitude, but the fact of the matter is
there is no formula or one simple trick.

~~~
civilitty
I fear that winning the genetic lottery is more than just a simple advantage.
In the case of being born into a wealthy family, it turns the situation from
"it's all up to you to succeed" into "all you have to do is not fuck up."
Since tremendous luck, good or bad, is relatively rare the condition from
generation to generation just doesn't change much - regardless of hard work -
except in so much as society itself grows wealthier.

------
curo
I've worked in a lot of coworking spaces where inevitably someone starts a
group to talk about the harsh realities of starting a business. It sounds like
Gambler's Anonymous.

Surprised to hear only 70% fail. Those odds actually sound pretty good. I've
made enough in the startup world to pay the bills for a decade, but probably
below my market rate at a big tech co.

I don't regret a thing. Each time I get a bit further into the wealth creation
game. But I also wouldn't recommend it to anyone, and actively work on
dissuading new founders from trying.

------
motohagiography
It is a relief to see this view reflected.

In an efficient market for business talent, VC should not exist. They could be
seen as in effect talent arbitrageurs. Something about business culture over a
certain size creates a massive inefficiency where someone with the level of
skill required to build a successful product is not snapped up by existing
firms.

Personally, I like anyone who takes an arbitrageurs view because to me betting
against bullshit is a costly signal of outlier intelligence.

However, viewed this way, it's not about kids in hoodies. College dropouts
were useful because the key signal (ivy certificate) that gets them picked up
by a firm that would smother their productivity, had been suppressed. That
they were selected for the school in the first place was a great negative
filter as well, so betting on the remainder is practically a no-brainer.

Given the exponential distribution of returns, from a statistical perspective,
there isn't much value in explanations for failure because it's by far the
most probable outcome anyway. e.g. who cares why startups fail, %90+ of them
necessarily must, and merely avoiding a subset of failure scenarios is not
equivalent to success. In that environment, there is only one necessary
condition for 80th+ percentile talent to maintain: survival.

Being the person who climbs up that return curve means surviving the twin
perils running out of runway, and of being snapped up by a firm for a job you
were originally more suited to anyway. This second peril is the other major
attrition factor that prevents people from benefiting from survivor bias.

The business of VCs who throw money at geeks in hoodies is to mitigate the
former and get exposure to that overlooked subset of the latter. Given the
remaining members of that sample, the people who also selected out of being
snapped up for jobs will be necessarily over represented - and on that very
steep curve. I think this explains why investors like colorful founder
stories, because they are (consciously or not) looking for how the founders
1-stddev+ talent signal might have been suppressed.

Good news is: the confluence of a) an exponential distribution of success and
returns, b) the guaranteed competitor attrition from failures, and c) the
further guaranteed competitor attrition from the job market - survivor bias
may be less of a bug, and rather, a feature.

------
jokoon
Are start uppers copying how Carmack started to do software? I don't
understand why there is so much need for raising money when all you need is a
roof and a computer to do this job.

I know that not everyone is talented like carmack, but sometimes I wonder if
money is not spoiling and troubling the IT sector.

So in fact a startup might think it failed because it doesn't have enough
money or because investors don't want to raise funds, but the reality is that
software is so cheap to make, I really don't understand the world of investing
in IT.

------
nathan_long
Survivorship bias is a problem in science, too, IIRC. Eg, "I looked for
evidence that eating onions causes cancer and found nothing" doesn't get
published, but the "successful" study does. I remember hearing of efforts to
make researchers state their intended research up front and publish the result
either way.

Similarly, shouldn't business schools should be eager to do studies following
new businesses, bootstrapped and funded, and analyzing their success or
failure? YC and bootstrapping communities could provide a lot of interviews if
they wanted to.

(This seems obvious and has probably been done.)

------
digilus0
I would like to see an analysis of startups that passed two years of age. That
sounds to me like enough time to learn how it feels and learn and experience
most of the issues in running a small business, and not enough time that it's
"wasted".

If it turns out that 70% of 3 or 4 year old startups fail, _then_ I'd say it's
a seriously bad tradeoff.

------
cushychicken
There's always this perverse focus on _who is making this thing_ , when there
should be a focus on _who is buying this thing_.

Seems like the number one indicator of success in many of these businesses is
_when people actually purchase or use the product_.

------
maehwasu
"In the same vein, a stunning proportion of partners at VC firms graduated
from a handful of tony universities, as if the seal on a person’s diploma were
what indicated investing abilities. (Granted, the incidence of leveraged
social connections and postgraduate degrees may amplify that trend.)"

Or it could be that these "tony schools" are primarily designed to select for
hyper-ambitious people with high IQs. Crazy, I know.

~~~
jimbofisher1
Considering around 1/3 of students at schools like Harvard are Legacy Admits
who would never otherwise get admitted on their merits, I don't think your
argument holds much weight. You seem to be admitting that they are selecting
for elite education, if they were selecting ambition and intelligence there
are countless people like that who went to a big State school instead of
Stanford yet they are not represented in "Elite" firms whether that's VC, IB,
Tech etc..

------
fourseventy
So was Elon Musk lucky in all four of his main ventures? (zip2, paypal, tesla,
spacex) Or can the luck factor be overcome if you are good enough.

~~~
joyeuse6701
I think previous failures and successes up your chances for success in
subsequent adventures. The failures teach. The successes teach and come with a
payout. The subsequent ventures make it easier to retain control with your
private money, as does the trust gained from previous successes, and your own
experience.

I think Musk has improved his chances of success by the principles above and
is improving his future chances by consolidating technologies in his
companies. Solar City, Tesla battery technologies, The Boring Company's use of
Model X chassis, The Boring company using the Hyperloop concept?

Of course, none of this would be possible without some runway, either
investors or private money.

~~~
slededit
There is more information in Success. If the likelihood of success is very
small you've potentially found at least some causal factor to help succeed -
the needle in a haystack. If you fail you've found one of the million reasons
you can fail. It's simply less informative.

------
jeffreyrogers
I guess there's some survivorship bias for startups if we define startups as
companies that take VC funding and intend to grow really fast, but those are
less than 0.5% of all companies. The fact is that people with domain
expertise, financial discipline, and a real product or service manage to do
quite well in business and the success rate is much higher (I believe around
50%).

~~~
edanm
Do you have a reference for that number? That sounds really high.

~~~
jeffreyrogers
Yes.[0] See item number 6. Anecdotally I know a number of people who have
started service businesses and they were all quite successful. They don't have
FU money and they probably have to keep working in some capacity, but they
seem pretty satisfied.

[0]: [https://www.sba.gov/sites/default/files/advocacy/SB-
FAQ-2017...](https://www.sba.gov/sites/default/files/advocacy/SB-
FAQ-2017-WEB.pdf)

~~~
edanm
Interesting!

I agree about some services businesses. Hell, I'm running my second successful
consultancy, and have been known to recommend to people not to pursue the
standard startup route.

Still, considering the sheer number of e.g. restaurant openings, small store
openings, etc, which I assume have a relatively high 5-year failure rate, made
me suspect the number was high.

I do wonder how your reference counts an establishment surviving after 5
years. E.g. if I start a company of 1 person (myself), and I keep it around
after 4 years, but I'm working a full-time job besides, I understand from the
reference that it would be counted as a firm that is still going after 5
years, although I wouldn't classify it as such. But I might be reading this
wrong.

------
diehunde
Now my question is, in all of those success cases, what was more important,
luck or work and dedication? Because if someone wins the lottery, it doesn't
matter if they bought 100 tickets every day. Luck would still be the main
factor of the success. Would also be the same for these companies like
Facebook and Google?

------
throw2016
You should be able to 'afford' to fail? If you can't afford to fail then the
risks are possibly too high.

It will likely set you back decades, destroy your finances and credit,
possibly impact your family life if you have one or further the possibility of
starting one and take a serious toll on your health.

------
pedalpete
I wonder if there are any stats around probability of success (measured by not
going bankrupt) at different levels of revenue?

For instance, what percentage of companies that are able to reach $5k monthly
recurring revenue go under then the same metric at $10k, $20k, etc etc

Anybody seen anything like that or have an idea?

------
nickstefan12
"How I won The Lottery" (joke tech talk)
[https://www.youtube.com/watch?v=l_F9jxsfGCw](https://www.youtube.com/watch?v=l_F9jxsfGCw)

------
zebraflask
The real problem is that 100K and some advice sets up the vast majority for
failure right out of the gate. Also weeds out the flakes, so there's that,
too.

