
When the U.S. air force discovered the flaw of averages - hecubus
https://www.thestar.com/news/insight/2016/01/16/when-us-air-force-discovered-the-flaw-of-averages.html
======
banku_brougham
The concept of no members of a group fitting into the average range for all
observations is reminiscent of the 'curse of dimensionality'. Can anyone with
a data science background make the connection here?

Secondly, this seems to explain why everyone hates autocorrect.

~~~
TTPrograms
Yeah, in this case if the 10 traits are independent and the chance of falling
in the "average range" is 1/3 for any one trait then the probability of any
one soldier falling the the average range for 10 traits is (1/3)^10 = 1/59049.

~~~
frankc
That is true, but this can be just as much an issue in 1 dimension. Consider
that the average person has roughly 1 testicle. I think it's about
understanding distributions, and joint distributions are a big part of that.

------
wpietri
The thing that strikes me here is how much we repeat the mistake of Norma
(taking the mythical average body and assuming that differences mean problems)
today in talking about people's minds.

I started using computers before it was fashionable, before my fellow nerds
started being worth billions and ending up on magazine covers. It's hard to
describe now how much our difference was seen as wrong, as the sign of a
problem.

I have to wonder how much other natural differences gets medicalized. I know I
have friends who take drugs for "insomnia" even though the only problem they
experience is that they don't always get the "average" night of eight hours
uninterrupted sleep. (Which anyhow is a modern invention. [1]) When I was in
mourning after my mom died, a few buttinskys suggested I talk to my doctor
about antidepressants, even though actual experts thought I was doing fine.
And I worry about the number of schoolkids who have their differences
medicalized because, in effect, they are inconvenient for overburdened
teachers using industrial-age models of education.

I have no solution here, but I definitely find it troubling.

[1] For more, see this podcast, especially the "Til Morning is Nigh" segment:
[http://backstoryradio.org/shows/on-the-
clock-4/](http://backstoryradio.org/shows/on-the-clock-4/)

~~~
Retric
Modern psychology does not care about average it cares about capable of
functioning in society. If you can't avoid say screaming in public that's a
problem not just a non issue. Arguably the tolerances might be low, but it's
hard to argue with the basic premise.

Granted doctors assume if your talking to them you have a problem. But that's
arguably a reasonable prior.

PS: The extremes of behavior get very far out there, notmenjoying talking to
people is fine, not being able to so communicate is an issue.

~~~
coldtea
> _If you can 't avoid say screaming in public that's a problem not just a non
> issue. Arguably the tolerances might be low, but it's hard to argue with the
> basic premise._

People are prescribed stuff for much much less than "screaming in public",
including feeling stressed in BS high stress jobs, being "hyper-active" at
school, etc -- with "modern psychology" more often than not siding with the
normality of abnormal and abusive societal norms (as it always did. Science is
conservative. Not so long ago you got medical treatment for being gay, for
example).

> _Granted doctors assume if your talking to them you have a problem. But that
> 's arguably a reasonable prior._

Doctors also prescribe and over-prescribe all kinds of BS to people who don't
need it, and get all kinds of pharma percs for doing so.

Heck, they'll go even to doing un-needed operations, compared to that,
prescribing BS is nothing:

[https://www.google.gr/search?q=uneeded+operations+scandal](https://www.google.gr/search?q=uneeded+operations+scandal)

------
keenerd
While hindsight is 20:20, parts of this should have been more obvious.

> Daniels generously defined as someone whose measurements were within the
> middle 30 per cent of the range of values

> Daniels discovered that if you picked out just three of the ten dimensions
> of size ... less than 3.5 per cent of pilots would be average sized on all
> three dimensions.

30% raised to the third power is 2.7%. Basic probability. I guess everyone
assumed there would be heavy clustering instead of largely independent
variables?

~~~
andrewfong
> The Aero Medical Laboratory hired Daniels because he had majored in physical
> anthropology, a field that specialized in the anatomy of humans, as an
> undergraduate at Harvard. During the first half of the 20th century, this
> field focused heavily on trying to classify the personalities of groups of
> people according to their average body shapes — a practice known as
> “typing.” For example, many physical anthropologists believed a short and
> heavy body was indicative of a merry and fun-loving personality, while
> receding hairlines and fleshy lips reflected a “criminal type.”

Also, for what's it worth, this was during (or at least not too far removed
from) a time period where researchers were trying to ascribe all sorts of
random characteristics to people based on whether they were "Negroids",
"Caucasoids", or "Mongoloids".

This says a lot to me about how people thought about physical characteristics
back then. If researchers assumed male-pattern baldness was associated with a
"criminal type", then it's not a huge stretch to imagine that researchers
assumed there was a "pilot type" with a large number of clustered or heavily
correlated characteristics.

~~~
foolrush
Lombroso et al.

Worth noting that this phenomenon is typically described as “medicalization.”
Othered groups medicalized include women, African Americans, homosexuals, and
many other groups. The medicalization of Jewish ancestry cropped up and was
amplified during a not-too-distant past.

------
trhway
joint distribution of 2 normal distributions isn't necessarily a normal
distribution in 2 variables. Independence would be sufficient condition for
that, yet obviously there is no independence between various measurements of a
human body.

[https://en.wikipedia.org/wiki/Multivariate_normal_distributi...](https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Joint_normality)

~~~
raverbashing
Which is funny, because the problem seems to arise exactly because the
variables are independent. If they were dependent then one could imagine all
sizes correlate with height, for example .

------
tokenadult
Wow! This has implications for human genetics (a study that originated partly
in considering human body size measurements like those described in the
article) that I think most popular writers on human genetics have not taken
into account. There was an article posted to Hacker News earlier this week
that suggested, surely falsely in my opinion, that human genomes can be
optimized for intelligence to such a degree that we won't even be able to
estimate IQ scores for future individuals with the same kind of tests that we
use now. I rather doubt it. Because of pleiotropic effects of genes, almost
surely there isn't a gene shuffle that will massively increase human
intelligence, but rather just strategies (as likely to be environmental as
genetic) to improve average intelligence within the range already found in the
worldwide human population.

Similarly, imputation of "race" by genome testing depends crucially on
assuming that the average genome is informative, but for any trait of
interest, a person who by both genome testing and known historical ancestry is
categorized in some "race" category might have any degree of variance from
"average" in the trait found in the whole human genome. It will be exciting to
follow up on larger and larger data sets on these issues as human genomics
projects continue.

~~~
yummyfajitas
_Similarly, imputation of "race" by genome testing depends crucially on
assuming that the average genome is informative,_

This is a nonsensical mischaracterization of both modern genetics and the
theory of high dimensional vector spaces.

The phenomenon the article is discussing is the fact that the mass near the
center of a normal distribution approaches zero as the number of dimensions
goes up, and that most of the mass lives a distance sqrt(N) from the origin.
This does not imply that a) you can't have gaussians or other distributions
which are separated from each other or b) that projections of one of these
distributions can't have a different distribution than a projection from the
other.

Please go read Foundations of Data Science, chapters 1 and 2 before expounding
on this topic further: [http://research.microsoft.com/en-
US/people/kannan/book-no-so...](http://research.microsoft.com/en-
US/people/kannan/book-no-solutions-aug-21-2014.pdf)

Can you clearly state what, precisely, you think this article proves about
either classification, intelligence augmentation or group differences in
specific traits?

~~~
shanacarp
I can talk about some of this, if only because of some of the ethical dilemmas
facing me and genome testing as of recent.

Racial Groups as described by a geneticist vs say, a government doing polling
on its citizens are different animals.

Goverments == mostly sociological. An example would be Hispanic in the US,
where growing up data collection would ask if someone was Hispanic, and now it
asks if you are white vs non-white Hispanic. Meanwhile in say, Meanwhile, if
someone crossed the border to Mexico, there is no such thing as Hispanic - you
can be White, Mestizo (Indigenous-European hybrid), Indigenous, and Other. It
is totally possible to live on the border of the US-Mexico, have reasons to
commute across the US-Mexican border, have citizenship to both countries, and
have totally different answers on your census depending on what country asked
you about your race/ethnicity, because as can be clearly seen, the way that
question is asked is different in mexico and the US to begin with.

Let's talk about being Mezito as an idea. From a geneticist's point of view,
it doesn't exist. (or at least, not yet, and it is unliekly to any time soon)
Which is how 23 and me and buzzfeed manages to get this Gif off of one of
Buzzfeed's employees, who very clearly feels he is half mexican (aka mezito)
[https://img.buzzfeed.com/buzzfeed-
static/static/2016-01/21/1...](https://img.buzzfeed.com/buzzfeed-
static/static/2016-01/21/16/enhanced/webdr01/anigif_enhanced-19351-1453410167-10.gif)
In other words, there are no clusters that define "mexican"/mezito

In order to have such clusters, you need to have long histories in one area,
histories of inbreeding, and other major causes to make mutations pop
selectively. Even with those mutations popping selectively, you also need
those mutations/genes to be very trait specific in most cases/ultra selective.
Otherwise, you are looking at junk.

So, one of the reasons ashkenazim are testable as ashkenazim is because they
have a very long history of inbreeding. So do the japanese compared to other
asian groups, same with the Finnish, especially if you are Saami.

[http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2575505/](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2575505/)

Still once these ibreeding/long history facts start to subside, you end up
with whatever is the strongest trait set in a pure mendelian fashion. It is
why there are plenty of ashkenazim who definitely decendants of the founder
populations of ashkenazim who don't carry other sets of Ashkenazi marker
genes, whereas there are ashkenazim where you can't tell who do carry these
other genes. The other random sets come from other places (sex..mutations..)
and are bred out with enough time. (Otherwise there would be many many more
questions about how I exist, genetically speaking, since I should not be able
to metabolize alcohol in about 1.5 hours...)

However, most traits are not one gene == one trait. It's how the music plays
together, especially in concert with epigentic factors. For many things
involving "intelligence" (or actually lots of things for humans), we're
looking at a non-mendellian inheritance pattern, including Genomic imprinting
issues (
[https://en.wikipedia.org/wiki/Genomic_imprinting](https://en.wikipedia.org/wiki/Genomic_imprinting)
). So while some gene related to "intelligence"/"thinness"/"insert something
here" might be more observable among certain groups/populations, it doesn't
mean it doesn't exist among other populations and some other factor, genetic,
epigentic, or otherwise, is suppressing your view.

This is why if/when you get involved with genetic research on humans, the
ideal is to get the entire family (as much as possible) involved in tests -
even if the original candidate comes from a human genetic isolate polpulation.
There are otherwise broader concerns about power and sample size in the study,
because you might be looking at something which is actually something else.

[http://www.nature.com/gim/journal/v4/n2/full/gim200210a.html](http://www.nature.com/gim/journal/v4/n2/full/gim200210a.html)

(but, hey, what do I know...I had to ask geneticists and people doing research
in this area when I found out my genome is worth something accidentally...)

~~~
nkurz
Great post, thanks! You use 'Mestizo' once, and then 'mezito' several times.
Is this slang, a repeated typo, or a term of the art? Google doesn't find much
for it.

~~~
barry-cotter
Mestizo means mixed Amerind European ancestry, exactly like mulatto means
mixed African European ancestry. They're both from colonial Spanish racial
categories.

[https://en.wikipedia.org/wiki/Mestizo](https://en.wikipedia.org/wiki/Mestizo)

~~~
nkurz
My phrasing was unclear. I'm familiar with "Mestizo" \--- it's "mezito" that
I'm wondering about. Urban Dictionary suggests that "mezito" is just a typo,
but the author uses it consistently and perhaps separately than Mestizo.

Deep in the search results are occasional other people who appear to be using
it intentionally:
[https://www.stormfront.org/forum/t53845/](https://www.stormfront.org/forum/t53845/)

I'm wondering if like "cholo" it has particular connotation:
[https://en.wikipedia.org/wiki/Cholo](https://en.wikipedia.org/wiki/Cholo)

Incidentally, I was recently fascinated to learn that some consider "cholo" to
come from Nahuatl:
[http://www.etymonline.com/index.php?term=Cholo](http://www.etymonline.com/index.php?term=Cholo)

------
TreeMan
My company Treemetrics has been challenging the forest industry for method
used for measuring forests. They use the average tree size as the method for
valuing a standing forest. From our data we see forests with thousands of
trees where only a handful of average trees actually exist. Very helpful
article for me to explain the flaw of averages.

------
ocdtrekkie
This is incredibly poignant in a world where our designers and engineers are
moving to provide users less options and less configuration, deeming their
ideal designs, meant to meet 'most users needs', is superior.

~~~
pacap
exactly!

------
ckib16
This was a very interesting article. And I can’t speak to the statistical
methodology described. But I think it paints too broad a brush in terms of why
the Air Force fatality rate (or “Class A Mishap Rate” as it’s called)
decreased over the decades. Quick side-note – I’m a former USAF pilot and
studied USAF history a bit.

The short version is – technical details like adjustable cockpits certainly
played a part. But the bigger driver was the ever-increasing culture of
safety, mishap prevention and leadership accountability.

Safety: things that were common place “back in the day” have been banned for
many decades in today’s Air Force. These include traveling without appropriate
weather checks, acrobatics maneuvers at low altitudes and unauthorized
airspaces, and flying without enough sleep.

Mishap Prevention: incredible resources are poured into scientific
investigations on what exactly happened in each incident so that A) we can
learn from it and B) all pilots can be briefed on the mistakes made so that
they can save their own lives.

Leadership Accountability: if you are a senior leader at an Air Force base
with a fatality, you have a large issue on your hands that needs to be handled
with extreme attention to detail. Lapses in safety are not looked favorably
upon at any level. I wasn’t in the USAF back in the day. But it’s fairly
obvious that there was less punishment meted out when mishaps occurred
compared to when they happen now. Different times – yes. But overall
fatalities were just accepted as “something that happens.”

The safety culture is all-pervasive and common in today’s military. But there
were different attitudes back in the ‘40s. So while many of today’s pilots
gripe about the excesses of safety culture - and I was one of them – most
pilots know deep down that it’s a good thing. I wouldn’t want to return to the
old school days where crashes were just the cost of doing business.

------
olalonde
Thought this would be about
[http://www.johndcook.com/blog/2008/01/21/selection-bias-
and-...](http://www.johndcook.com/blog/2008/01/21/selection-bias-and-bombers/)

------
my5thaccount
Articles like this remind me how rational and practical the military is. I
know so many brilliant military minds, yet war itself seems outdated and
plagued by so many emotionally irrational decisions. I can't reconcile those
two thoughts.

How can people inside the military be so incredibly smart, yet still think it
makes sense to ... you know, kill people's friends and families and not expect
them to become terrorists.

What am I missing? It can't just be greed. Military industrial complex. The
military minds aren't smarter than that? I struggle with it.

~~~
jballanc
I can't speak for the entire military[1], but I've known a few "brilliant
military minds" and they rarely, if ever, talk in terms of killing. The
military's goals are usually things like area denial, disrupting lines of
communication, limiting access to materiel and supplies, degradation of
morale, etc.

These aren't just euphemisms. If you're trying to win a war solely by killing
the enemy, you're going to be in for a looooong and _bloody_ war. In fact,
there's long been a thinking in the military that "a dead soldier removes one
soldier from the field, but a wounded soldier removes two".

What makes the military, and war, seem "outdated" or downright "despicable"
comes down to, I think, two things:

1\. War is often what happens when two sides have let long lingering issues
fester to the point that dialogue is not possible. In other words, the only
thing anyone wants less than to throw the first punch is to be unable to throw
the second.

2\. Yes, military industrial complex. Specifically, the MIC has muddled the
"goals" and abstracted them away behind many layers of "weapons systems" and
"advanced tactics". That is, if you're a general attempting to disrupt lines
of communication, and you have to decide on committing the lives of your
soldiers and potentially taking the lives of your enemy, you might consider a
battle plan that minimizes loss of life. If you're that same general and
Raytheon (or Lockheed or Honeywell or...) offers to sell you the CommsRuptor
7000 that will take out enemy communications at the press of a button (and a
signature on a check for $300M), you might not adequately question the impact
on lives. War used to be about loss of blood and treasure. Lately it seems to
be more about treasure and blood...

[1] Especially not the "grunts" that see front-line action. I've heard,
anecdotally, that enlisted soldiers are often trained in such a way that they
enter war zones with raw blood-lust. Whatever your opinion on this practice, I
took from your question that you were more interested in the thinking of the
decision-makers, i.e. commissioned officers.

~~~
Retra
I wouldn't say "raw bloodlust" is accurate. I'd say the life of a serving
infantryman is so boring, repetitive, and mundane that getting the chance to
use your training in combat will look very pleasing to almost anyone.
Especially on deployment.

------
bsbechtel
I'm pretty sure averages used in the wrong way affect many more fields than
any of us can realize today.

~~~
regularfry
That's basically the entire premise behind the Black Swan meme.

------
Sophira
For anybody getting a message that says "Please try loading the page again
after activating mobile data or connecting to a Wi-Fi network." = it's not
your connection, despite what the page says. The mobile site,
[http://m.thestar.com/](http://m.thestar.com/) , is giving that message on
desktop too and it comes from the site itself.

------
DigitalJack
Has nobody heard of quetelet?

~~~
cJ0th
I think we had a discussion here on HN not long ago but I can't find it.

edit:

Found the article:

[http://www.theatlantic.com/business/archive/2016/02/the-
inve...](http://www.theatlantic.com/business/archive/2016/02/the-invention-of-
the-normal-person/463365/)

comments:
[https://news.ycombinator.com/item?id=11155889](https://news.ycombinator.com/item?id=11155889)

------
brad0
Is it just me or does this sound oddly familiar to machine learning
recommendation systems?

ie: show an item based on the average of all the other items that everyone
else has looked at.

~~~
casperc
Yeah, I was thinking of the same thing. It's not my field professionally
though, so I am not sure what the relevant conclusion would be for machine
learning though (or if the comparison holds at all).

~~~
BillinghamJ
I don't think that's quite correct.

Recommendation engines normally work by taking the things you favor, then
looking at who else favors those things, and deducting that you're all likely
to have shared interests.

~~~
masklinn
Yeah I'd assume the simplest recommendation engines work on correlation. A
recommendation engine working on average would be complete garbage, you'd be
better off picking recommendation at random.

~~~
yeukhon
Quote emcq's response above:

> If you're thinking about the cold start problem when you dont have any
> information about a user, yes it's possible that your overall statistics is
> a combination of many subpopulations that doesn't really fit anyone very
> accurately but there are ways around this as well.

I'd say a better approach (but may have user experience penalty) would don't
even decide for the user, ASK. For example, take Flipboard / Quora as an
example, you are asked to choose some topics to follow at the beginning.
Assume research/data show 80% of the users are software engineers and 90% of
them always pick "technology" as a topic they want to follow, would you rather
show technology as one of the top five in the list of topics to choose?
There's actually a lot of experiments you can do from a simple
selection/survey process. I personally can't stand at going through pages to
find something relevant, but I am also surprised to find things I never
thought would be interesting to follow if I weren't present the options at all
/ or earlier.

------
dap
This is a great example.

It doesn't take a lot of dimensions, though. I've seen many system workloads
where the average latency was not the latency of any of the requests.

------
my5thaccount
Thinking about this and the wisdom of crowds makes my brain feel fuzzy.

------
daodedickinson
If the environment was homogenized enough this would no longer be an issue.

------
beachstartup
analogous anecdote: if you've ever tried to drive a sports car in a
seating/steering/pedal/shifting position that isn't tailored to your body, it
feels really awkward and it's very difficult if not impossible to drive fast
with precision.

i would guess that in a sports or race car, everything needs to be within a
few millimeters of where it "should" be in order for it to feel right.
multiply the speeds and divide reaction times by 4-5x and i can see how you
could easily crash a plane with a tiny margin for error.

------
j0e1
Reminded me of

“Be together. Not the same”

~Android :)

------
sandworm101
I don't see the problem. The fact that no person fit the model perfectly
doesn't mean that the model was a poor choice.

The question shouldn't be whether any customers fit the cockpit perfectly, but
how many customers do and do not fit absolutely. You build the legroom so that
most legs will fit, and headroom so that most heads will fit. Then as many
people as possible shall fit. Everyone will have some dimension that isn't
perfectly accommodated, but few should be rejected. The fact that nobody fits
perfectly doesn't take away from a design that reasonably accommodates as many
people as possible.

A cockpit is not a suit. It's a communal chair/workspace meant to be used by
various persons over many years. The metric of a good design should be how
many/few people are so out of standards that they cannot work properly in the
space. This is a perfect metaphor for hiring. Look only for that 'prefect fit'
and you won't hire anyone, or you end up with hiring the only applicant to get
past the roboreader. Seek a broader standard and you'll find plenty of good
people even though mr perfect never appears.

~~~
throwaway_exer
> A cockpit is not a suit.

The opposite is true. To fly an airplane with precision, you need to "strap
the airplane on."

That means sitting in the position with the best visibility and best reach
inside the cockpit, even a Cessna 172.

(In a 172, you should sit high enough to be able to see each of the rivets on
top of the cowling.)

~~~
sandworm101
Yes, but the seat does adjust. It isn't the case that the cockpit must be
tailored to every person. Each Cessna, 747 or f-16 seat is practically
interchangeable amongst the type. These aren't WWI biplanes with nothing more
than a cushion. Even then, the cushion height, the prime dictator of eye
level, is adjustable.

~~~
vlehto
I don't know about postwar planes, but at least some of the WWII war birds had
notoriously poor seat adjustments. For example BF-109 had a seat that doubled
as armored fuel tank. It's bit tricky to add adjustments to such heavy piece
of furniture, so they didn't.

The thing here, just like you pointed out, is not to tailor the seat for
average nor individual, but a _range_ that suits most individuals.

