
"Attrition is a group statistic. It doesn't apply to me." - arram
http://josephwalla.com/attrition-is-a-group-statistic-it-dont-apply-to-me
======
RyanZAG
Special forces training is taken by a large number of people - some of these
people are just chancing it, hoping they will be able to push through anyway
but not really prepared or ready for it. Others, like your brother, are fully
prepared. When looking at a base rate, it's important to understand the
different divisions inside that base rate and the reasons people drop out.

In special training, the reasons people are likely to drop out are because
they aren't physically fit enough to cope. This gives your a brother a good
indication - he likely knows others of similar fitness to him who passed. The
key here is that there isn't much randomness involved - either you are fit
enough or you aren't.

Startups are a completely different ballgame. The main factors behind dropping
out are a complete mystery - team composition, relationships, lucky deals with
investors, hitting your marketing in at the same time some other big player
pulls out or pitches in - the list is endless and impossible to even fully
understand.

Basically what I'm trying to get at is that comparing something you have
complete control over (special forces training) vs something you don't (a
startup's success) is not a great analogy.

However you're still correct anyway - the best indicator of whether a startup
succeeds is whether or not they try in the first place! So it's always best to
try and ignore the baseline.

~~~
auctiontheory
> In special training, the reasons people are likely to drop > out are because
> they aren't physically fit enough to cope.

That is the opposite of everything I have read about special forces training.

Anyone who makes it past the initial stages is extremely physically fit. The
difference between the qualifiers and the rest is in the mind.

~~~
dclowd9901
Still, it's in your mind. To make it more like the startup world, the gauntlet
should randomly throw in bear attacks.

------
calinet6
Aww. I bet the person who won the lottery thought they were special, too.

Clearly, there is a grand misunderstanding of statistics here.

In order to make yourself an exception to the statistics, you're going to have
to scientifically explain how you are going to become better. Everyone
_believes_ they're better. How are you, truly, going to go above and beyond?
Do you have better connections than the average? Do you have the experience
needed? Do you have a better idea which is downright guaranteed to succeed?

If you can answer Yes to all of those questions, congratulations, you are the
average Startup Individual, and the statistics do apply to you. You're going
to have to work really hard, have all the right things fall into place, and do
everything right, and even then you still have an extremely large chance of
failure. It is the nature of the situation.

Starting a startup isn't like playing the lottery. You have maybe 20% more
control over your outcome. But 80% is still in the hand you're dealt, both as
a person and as a company. Make the best of it.

~~~
samspot
I don't think it is a misunderstanding. For example, the overall rate of heart
attacks could be something like 10%, but that doesn't mean that I personally
have a 10% chance of having a heart attack. My actual chance is likely higher
or lower depending on my family history and other factors. Group statistics
give us valuable information, but they are not the whole story.

~~~
kybernetikos
People usually incorrectly assume that they are special. While it's certainly
possible that there are extenuating circumstances that mean you aren't going
to have a heart attack, in the absence of really strong evidence pointing in
another direction, it's more rational to assume that the group statistic does
apply to you.

~~~
dasil003
One isn't a scientist observing one's own life. Being a realist is a good
principle, but rationality is not an end unto itself. In the field of human
endeavour, mental attitude is a huge factor. You can weigh out different cases
in terms of odds, but the actor who dwells upon statistical improbability will
have a much smaller chance of success than the one focused on actually
achieving the goal despite the odds.

~~~
kybernetikos
It's true that mental attitude is important, but you can still take a rational
approach to situations like this. Look honestly at the odds. If you accept
them, then cultivate the appropriate mental attitude to help you succeed, but
if you do not start with a realistic assessment of the true situation then you
are likely to waste much effort trying to replicate the paths that outliers
have taken, effort that could have been better spent.

That realistic assessment should start by assuming that in the absence of
further data you are a typical member of your class. If you have ideas about
ways in which you are different to your class, try to validate them with
actual data before making judgements based on them, because another feature of
typical members of the human race is that they frequently invent spurious
reasons to believe they are atypical.

------
dvt
Contrary to most commenters here, I've recently started to embrace "being a
statistic." I think completely understanding where you stand (on the normal
curve) is not only important when helping you prepare for taking an exam,
approaching a girl, or launching a startup, but also important for the many
(many) inevitable failures. You will be able to deduce what you did right,
what you did wrong, how to improve, etc. given where you started.

I think that embracing your inner statistic is a good, if not great, idea.
Don't be depressed because you're not that special snowflake, instead try to
understand where you are, where you want to be, and what, exactly, you have to
do to get there. If there's a 99% chance that you won't get from where you are
to where you want to be doing whatever it is you're doing, either amend your
goals or do it a different way.

Sometimes there's no shame in folding.

~~~
kybernetikos
> Sometimes there's no shame in folding.

Actually, a correct fold is one of the most skillful things you can do in
poker. That's why James Bond will never be a good poker player.

~~~
ryguytilidie
Why would you ever fold if you always have the best hand?

------
cletus
His brother is correct: statistics are a group property, or rather, they are
they description of what happened with a particular group.

When when you start making predictions you're talking probability not
statistics. Statistics may inform probability but there can be issues with
this (eg bad sample, insufficiently sized sample, failure to identify the
underlying characteristic(s) such that samples are actually flawed and so on).

The problem we have with a species is that we conflate the two. This comes up
with highly political issues such as racial profiling. Imagine a situation in
a southern border state where statistically more illegal aliens are Hispanic
in both absolute and relative terms (meaning absolute number and relative
percentage of their given populations). I say this not as a fact but as a non-
judgemental scenario to make a point: law enforcement will likely, consciously
or subsconsciously, make different suspicions based on such external
characteristics.

Is this fair to the individual? Of course not. It borders on an assumption of
guilt. People of Middle Eastern descent who regularly fly I'm sure are cursed
with this kind of suspicion when almost all of them have done absolutely
nothing wrong.

Anyway, as to special training, a relative of mine was married to a guy who
went through SAS training (and passed). To those who think this is simply an
issue of being sufficiently physically fit, you couldn't be more wrong. Now
you have to be sufficiently fit for it to be possible to pass but after that
it's a mental issue.

They put these guys through mentally strenuous situations: sleep deprivation
followed by interrogation, psychological profiling and the like.

His brother has another point though: on an individual level you should never
assume you're going to fail (or succeed for that matter) based on group
statistics. Engineering is currently male-dominated but no woman should ever
take that as evidence that they can't do it as an individual. Nor should any
man assume a specific woman is less likely to be capable, etc.

~~~
gambiting
You don't even need racial profiling to show that statistics harm individuals.
Look at car insurance - in most countries you will have a massive premium to
pay,just for committing the crime of being under 25. Even if you are 24, had
your licence for 6 years, drove hundreds of thousands of miles without an
accident, your insurance will still be higher than that of a 26 year old
person who has just passed their drivers test and doesn't know anything about
driving. Why? Only because "statistically" you are more likely to be in an
accident. Well, so are men, yet it is illegal to make their car insurance
higher because of their gender in almost all EU countries. You may not have
done anything wrong, but are still "punished" for being part of the statistic.
It makes me really upset sometimes.

~~~
nostrademons
Well, they also help individuals. Think of the person who is 26 years old, has
just passed their drivers' test, and then totals their car and kills the other
driver. Insurance shells out a few million in liability claims. Their
insurance will go up, obviously, but they're not going to be financially
wrecked, and it's the rest of society that bears that burden.

The way to think of it is to think of your whole life statistically, as a
series of events where you win some and you lose some. And then don't get
upset over the ones you lost, because there are many others where you ended up
with the winning end of the deal and you just don't remember it because humans
are wired to dwell on losses more than gains.

~~~
gambiting
Sorry but I just cannot agree with your first statement - even if an 18 year
old driver causes a crash and kills the other driver, the insurance company is
still going to pay 100% of the claim, unless the claim goes over the insurance
limit on your specific policy(which in EU is unlimited for death and bodily
injuries). So I do not see how paying more when you are under 25 is a benefit.

------
coldtea
> _He explained that statistics are created for groups, not for individuals.
> As an individual, it's not relevant to you. He went on to pass the course on
> a broken ankle. That makes me think about some startup founders. Their
> companies should have died, since most die. They should have given up, since
> most give up. But, somehow they willed their company to exist. Startup
> failure is a group statistic, it doesn't apply to you._

Yes. Also known as "survivorship bias".

Fact is, the brothers of all the other people who attended that course could
have (and might had) said the exact same things to their brother. And most of
them would be proven wrong.

Group statistics are indeed about groups. It's our way to predict what the
most likely outcome is for individual members of a group taken at random (or
all other things being equal).

~~~
unoti
>Fact is, the brothers of all the other people who attended that course could
have (and might had) said the exact same things to their brother. And most of
them would be proven wrong.

Exactly. But those people don't talk about it a decade later. And that's
selection bias.

------
PaulHoule
This is one of the Five Hazardous Attitudes that make you an accident-prone
person, a risk factor identified by the FAA

[http://www.avhf.com/html/Evaluation/HazardAttitude/Hazard_At...](http://www.avhf.com/html/Evaluation/HazardAttitude/Hazard_Attitude_Intro.htm)

Reduce your risk and improve your expectation values by rationally using
decision theory and correcting the systematic sampling biases in your head.

~~~
quaunaut
No, it isn't. This isn't him saying, "It won't happen to me" just because,
it's that it doesn't apply because he's taking the necessary precautions. He's
doing everything he can- and other people won't affect HIS rate of failure.

~~~
calinet6
Yes, they will, and that's what makes him failure-prone.

When outside influences do come along that affect his expected outcome, he
will be disrupted instead of scientifically understanding the situation, and
his role as an individual who is not the center of the universe.

~~~
quaunaut
You are way too deterministic.

~~~
PaulHoule
No, I'm a Bayesian.

When you know there is sampling error (luck) it makes sense to blend a little
bit of the distribution of the group in with each individual.

For instance, if you look at baseball hitters for a season ordered by batting
average, probably the guy at the top of the list got lucky (sampling error)
and the guy at the bottom of the list had bad luck. The real batting average
is a hidden variable that we can only see through sampling.

A good estimator of an individual's batting average can use the group
distribution as a prior against the individual distribution as an observation
-- this regresses the individuals back towards the mean, which is a realistic
way to perceive uncertainty.

~~~
monkeyspaw
As a baseball stat head, I can extend your analogy.

In late 1990s, when Quinton McCracken first published DIPS theory, it was
postulated that pitchers had zero control over their "luck based" factors
(BABIP, in this case; expanded to include LD%, LOB%, HR/FB%, etc.)

Now there exists enough data to identify outliers, to whom the theory of DIPS
(Defense Independent Pitching Stats) doesn't apply. Matt Cain, for example, is
acknowledged to outperform his DIPS.

No matter what the statistics suggest, people will believe they're the Matt
Cain. And some will be. By definition, they're the outliers that our
statistical _models_ do not account for.

------
crntaylor
If you put your mind to it, you can achieve anything.

Except compute an uncomputable function. Or transmit information faster than
the speed of light. Or simultaneously measure the speed and momentum of a
particle with errors whose product is less than half the planck constant.

But apart from that, anything.

~~~
crazygringo
Do you really believe that, or are you being sarcastic?

Because I hate to tell you, someday you'll collide with real life. If you put
your mind to it, can you become president? Chances are simply, no. A lot of
people have put their mind to it, like Romney, McCain, Hillary, Kerry... it's
not like they were lacking in motivation or desire.

So... no, sometimes, no matter how hard you want something, you are not going
to achieve it. I know there's a big self-help movement, believing in
attracting positive energy, believing in yourself, and whatnot... but a lot of
people can't seem to always tell the difference between realistic expectations
and self-delusion.

~~~
crntaylor
> _Romney, McCain, Hillary, Kerry_

They probably just didn't want it enough. If they'd really thought positively,
if they'd just visualised themselves in the position they wanted to be in,
they'd all be president now.

And if you're going to say "but what about their opponents?". Well, if they
believed in themselves enough, then they'd be president too.

If Romney and Obama both believed in themselves, then they'd _both_ be
president.

Now _that's_ the power of positive thinking.

~~~
delluminatus
So you were being sarcastic.

Having said that, positive thinking and motivation are very important factors
in goal achievement. It's often the case that when someone fails to accomplish
something, it's because there was simply something else more important to them
that inhibited their effort.

~~~
swombat
In other words, positive thinking is very helpful, possibly even a necessary
precondition for some outcomes (like becoming president - ain't gonna happen
unless you really do want it), but it is neither a predictor nor a principal
factor.

On the other hand, it is arguable that negative thinking can be a fairly
strong predictor of failure. That is perhaps the most useful take-away from
the otherwise relatively useless field of positive thinking: if you keep
thinking you're going to fail, keep visualising and expecting failure, chances
are very good that you will indeed fail, even if you had all the other
necessary factors for success.

------
caseydurfee
Reminds me of Steven Jay Gould's classic "The Median Isn't the Message":

"If the median is the reality and variation around the median just a device
for its calculation, the "I will probably be dead in eight months" may pass as
a reasonable interpretation.

But all evolutionary biologists know that variation itself is nature's only
irreducible essence. Variation is the hard reality, not a set of imperfect
measures for a central tendency. Means and medians are the abstractions.
Therefore, I looked at the mesothelioma statistics quite differently - and not
only because I am an optimist who tends to see the doughnut instead of the
hole, but primarily because I know that variation itself is the reality. I had
to place myself amidst the variation."

<http://www.cancerguide.org/median_not_msg.html>

~~~
calinet6
He certainly had a way of bringing out the beauty in truth.

What he's saying here really is useful, since he's also recognizing the
variation. He knows that he is not the median or mean, but a point somewhere
in the field among other points. The median does not exist; it's only an
abstract measure of the wide range of variation and commonality.

However, it would be foolish (or, as he puts it, optimistic) to place yourself
blindly at the top of the bell curve without evidence to support that.

As Gould (bless his heart) says in the paragraph following that: "I possessed
every one of the characteristics conferring a probability of longer life: I
was young; my disease had been recognized in a relatively early stage; I would
receive the nation's best medical treatment; I had the world to live for; I
knew how to read the data properly and not despair."

He was a scientist. He had _evidence_ and reasons for believing he lied where
he did among the statistics. He didn't _choose_ to believe that, he _deduced_
it.

If someone has good reasons for believing they're going to be better than the
average, more power to them. They're simply making an intelligent conclusion.
But you can't choose to be successful by sheer willpower alone. You have to
make a case for it; build evidence that proves it. I don't see that here. I
don't see it with most people who believe this sort of anti-statistical
nonsense. All I see is a misunderstanding of statistics; a lack of respect for
the variation and where you are likely to lie within it.

You have to understand that before you try to do better.

------
jiggy2011
You still need a backup plan, I assume that if you fail special forces
selection you get to keep your job in the regular military.

If your startup goes bad, make sure it doesn't leave you in an untenable
financial situation regardless of how optimistic you are.

~~~
comrade_ogilvy
That is actual wisdom. It is okay to lie to yourself about your odds of
success, if you have done your homework and have a viable Plan B.

Having the "best plan" is usually only possible in hindsight. My observation
is that the most successful people are not necessarily the ones with the best
plan, but those who act decisively on good plans and know how to pick
themselves up with an even better attempt after a failure.

I do not really see how failing to enter special forces has a huge downside.
Even not succeeding offers the opportunity to learn.

------
justin_vanw
Although I like the sentiment, I disagree with the accuracy of this.

Attrition is a group statistic. It doesn't apply to you if and only if you are
exceptional in some way. Otherwise, it applies to you even more.

In the specific example, his brother knew he was so much harder than his peers
that he could do something incredible, like run for miles with a broken ankle.
That mental toughness is how he was exceptional, and why he could be confident
that he would be one of the people that made it.

~~~
kybernetikos
You're probably giving him too much credit. I've seen interviews with people
before they embark on these kinds of courses, and every single one of them
said that they really wanted it, that it was the most important thing in their
lives, and that the group statistic didn't apply to them.

Some of them made it. Many of them didn't.

------
Kylekramer
Ignoring base rates is a great idea until you are hit by a base rate. And by
definition, you are probably going to be hit by a base rate. Everyone who
succeed _and_ failed all thought they were an special individual.

------
thetrumanshow
Most people don't win the lottery. Statistically speaking, you almost always
lose. So why play, right?

Sometimes, its okay to ignore the statistics and hope for the best. Sometimes,
its okay to stop hoping and start believing, even if the odds are grim.

Selection bias does teach us one useful thing. That you don't win if you give
up.

~~~
garry
The lottery is one where everyone pretty much has equal odds of success.
Individuals, especially in startups, tend not to have equal odds at all. It's
a game of skill, not luck.

~~~
jpdoctor
> _It's a game of skill, not luck._

I'm gonna guess that you didn't found a company at the Nasdaq peak in April of
2000.

------
jeremyjh
I'm sure most people in his selection group felt the same way, and most of
them failed. We only write blog articles about the exceptional ones.

------
lelandbatey
I feel like not being realistic about your odds is a very dangerous game to
play. Sure, you can _maybe_ grit your teeth and bear it. But you also might
totally burn out doing something unrealistic.

This applies to more than just start ups: It's very rare that you can succeed
through sheer stubbornness, many times you'll just end up hurting yourself.

~~~
benevpayor
I think he's trying to say that there's a difference between _your_ odds and
the population odds. The key is to know thy self.

------
utopkara
Startup success and lottery wins are very different things. You don't get
better at lottery as you buy more tickets over the years; but you get better
at your trade as you do it more and more, even if you fail at each previous
attempt to start a company.

If you see it as a lottery, then it doesn't make sense for people to try
starting their own companies after several failed trials when they reach some
middle age, as they really don't have much time left. But, it is just the
opposite, because they are actually much better at seeing pitfalls and bs, and
are a lot more likely to be successful.

So, the statistics do matter, you just need to know the kind of statistics
that apply. As a rule of thumb, in real life, it is usually not the lottery
kind.

------
dools
Classic survivorship bias scenario. Nevermind that all the other people in the
course went in with equally inspiring slogans like Failure Is Not An Option
tatooed across their buttocks. Some people fail, sometimes it's going to be
you.

------
jes5199
We're doing motivational posters here now? Okay, we can do that.

I recommend: the Tao of Programming <http://www.canonical.org/~kragen/tao-of-
programming.html>

~~~
mappu
Related reading: The Codeless Code <http://thecodelesscode.com/contents> and
ESR's Rootless Root <http://www.catb.org/~esr/writings/unix-koans/>

------
shokwave
60% of people drop out[0] - before you knew this, you had an 60% chance of
dropping out. Now you know the statistics, you can go about exerting effort to
end up in the 40% that don't drop out. And that number - 60% to 40% - tells
you a little bit about how much effort you're going to need to put in (not
much).

90% of startups fail[1] - now you know that, you want to put yourself in the
10% that don't. 90% to 10% is a bigger gap to jump than 60 to 40, so it's
going to be a lot harder.

0,1: not actual statistics

------
edanm
This is The Biggest Lie Entrepreneurs Tell Themselves (tm).

You hear about the fact that only 10% of startups succeed, but you tell
yourself "I'm better than them". pg himself reinforces this notion by saying
things like "some founders have a 0% chance of success, others a 95% chance".

pg is right, in a sense, but there is very little way for you to know whether
you really are "better" than others. Very few people actually know what makes
a startup successful, so there's that. On top of which, the people who are
taken into account in the 10% statistic includes, among others, people who are
building their 5th startup, the sons of highly influential politicians,
personal friends of movie actors, etc. It includes people 30 years in a
particular industry, with tons of insider knowledge, who decided to build a
product in that industry. And it includes other 21 year olds with stars in
their eyes, and nothing more.

This isn't to say "don't build a startup". That's a different conversation,
and depends a lot on your goals in life.

But most people are in the startup game at least partially for the money. And
for the purposes of calculating your potential earnings, unless you have a
_good_ reason why you're special, _pay attention to the damn statistics_!

------
specialist
My oncologist told me "Statistics are not people."

I was about to undergo a bone marrow transplant. At the time, mortality within
the first year was 90%. But I was much healthier initially than most other
patients, because I had different medical history.

Plus, I'm one tough SOB.

~~~
_delirium
> But I was much healthier initially than most other patients, because I had
> different medical history.

That just seems like you need better statistics. More sophisticated cancer-
prognosis models typically do take initial health and medical histories into
consideration.

------
Detrus
What if he broke more than his ankle?

Obviously he was mentally and physically prepared for the course. But a course
is predictable. Startup economics are more like combat, you can be the best
soldier but an IED finds you.

------
MattBearman
"Who cares what statistics show? Look at medicine. 80% of people with
pancreatic cancer die within 5 years and 95% of appendectimies occur with zero
complications. But we both know cases of pancreatic cancer patients that lived
and appendicitis patients that unfortunately passed. Statistics mean nothing
to the individual" \- Dr. Cox (Scrubs)

That quote has always stuck with me since I saw that episode, if you enter a
situation knowing statistics of outcomes, it can affect how you perform.

~~~
sopooneo
And maybe it _should_ affect what (if not how) you perform. If you are
probably going to die soon, maybe you would feel your life best spent in a
different way than if you had decades left.

------
D-Train
Meh, I agree with him... if only because I'm one of those guys who cares about
how I perform. It's like being in the gym... I'm not going to be the strongest
guy in the room, but what I do care about is how progress.

At the end of the day, I don't care how others do it outside of learning from
how others didn't succeed and those who did.

------
_delirium
Between this article and the beekeeping one, HN isn't making a good showing on
the day for scientific literacy.

------
bjliu
"...we've found a very serious problem: no matter how carefully we select the
mean, no matter how patiently we make the analysis, when they get here
something happens: it always turns out that approximately half of them are
below average!" -Richard Feynman

------
abc_lisper
This is a very acute observation. It applies to you, but not in a way you
think. In fact, as an individual it probably means the opposite to you.

Check out Douglas Hofstadter's GEB, for a really enlightening take on the
topic. Check index for "Aunt Hillary"

------
phil
This is more or less the same point that Stephen Jay Gould made in "The Median
Isn't the Message:"

[http://people.umass.edu/biep540w/pdf/Stephen%20Jay%20Gould.p...](http://people.umass.edu/biep540w/pdf/Stephen%20Jay%20Gould.pdf)

------
cbsmith
Classic case of not understanding statistics and probabilities...

------
soup10
Extreme determination is a rare and powerful thing.

Combined with some talent, luck, humility and hard work, one can accomplish
anything.

------
nezaj
I've always viewed statistics as information about aggregates. Anything can
happen when N = 1

------
baqbeat
You are not a special snowflake.

------
WalterSear
So is survival rate.

------
michaelochurch
This is why there are so many graduate students who can't find jobs. The
prospects (acceptable for STEM, dismal for all else) are published, but people
who are used to being the "1 in 10" fail to realize that, when it counts, they
likely will be. The other 9, it turns out, are Ivy-educated smart people just
like them.

This might apply to VC-istan as well, although I think founders are less
delusional about it than employees. Founders become EIR if the thing flops;
employees think the business is "fully de-risked" (because they're told that
to justify the 0.0x-percent equity slices) and that they're _guaranteed_ to
actually get the executive positions they were promised (ha!).

