
Beware Isolated Demands For Rigor - spindritf
http://slatestarcodex.com/2014/08/14/beware-isolated-demands-for-rigor/
======
cadlin
Since this seems to be the overall point

>But when other people are totally happy to talk about speed and blood
pressure and comas and the crime rate, and then suddenly switch to a position
that we can’t talk about IQ at all unless we have a perfect factor-analytical
proof of its obeying certain statistical rules, then I worry they’re just out
to steal cows.... But when people never even begin to question the idea of
different cultures but make exacting demands of anyone before they can talk
about different races – even though the two ideas are statistically isomorphic
– then I think they’re just out to steal cows.

There is of course, the opposite problem, which is that people take things
like IQ and infer wild conclusions, usually in the service of white supremacy.
An example of this is the MAO "Warrior Gene." There are several studies
showing a weak correlation between a variant of the gene and violence. Black
Americans have this gene more often.

There is a specific, very bad, study that is often quoted. It cherry picks its
data, and still can't achieve a significance of p > .05.[1] Yet it is often
brought up as indisputable proof of the savage nature of black people. It's
ridiculous, and IMHO, much worse than the tendencies criticized in the blog
post.

[1]
[http://www.soc.iastate.edu/staff/delisi/MAOA%202013.pdf](http://www.soc.iastate.edu/staff/delisi/MAOA%202013.pdf)

~~~
api
Side note...

Funny thing about white supremacy. If you go on raw IQ statistics alone, then
whites are certainly not the master race. That title would belong to Asians
and Ashkenazi Jews.

But then again I'm a skeptic of IQ = everything.

~~~
tormeh
It's a benchmark for the brain. It's not close to everything, but it measures
something and that something is significant. I'm sure you agree, but I need to
get it out.

~~~
wavefunction
IQ is only significant within its own context and that significance is
functionally limited even within that context: an individual requires an IQ
above a certain threshold to not be considered "developmentally delayed" but
above a certain threshold, it also doesn't mean much.

By context, I mean that IQ has meaning for the way life is for humans right
now, but given a return to an earlier context (hunter-gatherer) or a
transition to a new context (say some high-tech utopia where computers do the
heavy thinking), it becomes functionally meaningless.

~~~
barry-cotter
You've been Gladwelled man.

[http://neoacademic.com/2011/10/05/gladwell-was-wrong-high-
an...](http://neoacademic.com/2011/10/05/gladwell-was-wrong-high-and-very-
high-ability-employees-perform-differently/)
[http://pss.sagepub.com/content/22/10/1336](http://pss.sagepub.com/content/22/10/1336)

The nature of the relationship between ability and performance is of critical
importance for admission decisions in the context of higher education and for
personnel selection. Although previous research has supported the more-is-
better hypothesis by documenting linearity of ability-performance
relationships, such research has not been sensitive enough to detect
deviations at the top ends of the score distributions. An alternative position
receiving considerable attention is the good-enough hypothesis, which suggests
that although higher levels of ability may result in better performance up to
a threshold, above this threshold greater ability does not translate to better
performance. In this study, the nature of the relationship between cognitive
ability and performance was examined throughout the score range in four large-
scale data sets. Monotonicity was maintained in all instances. Contrary to the
good-enough hypothesis, the ability-performance relationship was commonly
stronger at the top end of the score distribution than at the bottom end.

~~~
wavefunction
I haven't been Gladwelled as I avoid the guy and this is my own opinion formed
from being around a fair number of very smart people over the course of my
life. In my opinion, IQ matters to a point, "good-enough" if you want to call
it that, and then other personality factors or strengths and weaknesses take
over.

~~~
om2
Does the cited study affect your opinion?

------
dllthomas
While I agree with the admonishment (see also
[http://lesswrong.com/lw/km/motivated_stopping_and_motivated_...](http://lesswrong.com/lw/km/motivated_stopping_and_motivated_continuation)),
I think the PBS example is bad.

The question of how we trade off different methods of achieving broadly
humanitarian ends is a different question than how we trade off broadly
humanitarian ends against different ends. If the only case being made for the
air strikes is humanitarian, then it makes sense to see if it's reasonable
humanitarian bang for the buck. Of course, there are other ends that the air-
strikes might serve, though I have no sufficiently informed opinion on their
effectiveness (or counter-productiveness).

------
aristus
This is a tactic that I've yet been able to counter effectively. It's more
about mixing levels of rigor.

Case in point: a tech company releases diversity numbers. A reasonable point
is to ask for a breakdown into tech, admin, support, etc. Often in these
companies the diversity is shoved into sales and marketing.

But then another commenter will ask about the diversity of the candidates, and
speculate that they are identical to the company's hires, and then "prove"
that the company's practices are unbiased.

So then you point out that a company is not a blushing maid, that they choose
the people they recruit, project an image that influences self-selection, and
heavily encourage people to refer their friends. And then you get attacked for
demanding too much rigor.

What the argument is really about is whether there is a problem to fix. The
diversity numbers clearly indicate that there is. But people who like to
believe there is not alternately dig trenches and build walls of rigor to
block the debate.

~~~
yummyfajitas
_What the argument is really about is whether there is a problem to fix. The
diversity numbers clearly indicate that there is._

You can't draw a normative conclusion from a positive claim alone. You
additionally need some normative axiom. No fact can tell you what you _should_
do.

In the discussion you cite, the disagreeing parties have different normative
axioms. Some people believe in statistical equality as an end goal, whereas
others have equal treatment. If you believe in statistical equality (as I
think you do), then the facebook numbers are a problem to fix. If you believe
in equality of treatment (as I suspect the others do), then you can draw no
conclusion - the numbers do not directly imply anything about equal treatment.

Similarly, consider a debate between a "meat is murder" activist and a
carnivore. No one disagrees that animals are killed - the only disagreement is
over whether that's a bad thing.

~~~
aristus
A good example of the debate and tactics I'm describing. :)

If you agree that women are not somehow less mentally capable than men, then
the fact that technical jobs are 85% dominated by men should be a concern. Not
just for "normative" reasons or "statistical equity", but simply that there
are a lot of talented people being excluded for one reason or another. If not,
you should explain WHY not. Anomalies demand explanation.

You dismiss the numbers because of an assumed equality of treatment and
saying, well, I don't see anything therefore it's not a problem... and if it
is, it's somebody else's problem. That's exactly the mismatch of rigor I'm
talking about. You put the burden on others to "prove" that something is
amiss, then when they do, demand more.

~~~
yummyfajitas
Claiming that "a lot of talented people being excluded for one reason or
another" is a _positive_ claim. It's merely a fact about the world. If you are
against this, that's a _normative axiom_ (or possibly a normative conclusion
derived from another normative axiom).

As for the undue rigor involved, it's you who seems to be demanding a lot: you
demand proof that men and women are statistically unequal merely to admit it
as a possibility.

More precisely, let A = "women are unequally treated" and B = "women are not
statistically equal to men" (at the point of application for Facebook
employment). The diversity numbers imply A || B. You are assuming that B is
false because there is no proof it is true, and therefore concluding A must be
true (since A || B and !B => A). But strangely, you are not asserting that A
is false without a rigorous proof that A is true (which would imply B).

That's the fundamental asymmetry that Scott Alexander is talking about. The
people you disagree with are merely saying "A || B, we don't know which."

~~~
aristus
"We don't know which", and will stop there. That's fine. All I ask is you be
explicit about where you stand: on the side of the status quo, not wrapped in
the flag of math as you would like pretend, but simply content with the way
things are, and utterly incurious about how to change it.

[edit] For fuck sake - if you had noticed a 50% imbalance in CPU usage on your
server farm, surely you'd spend some time trying to understand it. What the
hell is wrong with so many people who are busy inventing facile excuses and
dredging up mathy reasons why to NOT recognize the problem as a problem and
looking into why it might exist?

~~~
yummyfajitas
If I narrowed the source of CPU usage down to this block of code:

    
    
        for {
          x <- f()
          y <- g()
        } yield (x+y)
    

I would not immediately conclude that f() is the source of the problem. That's
what you are advocating doing.

And if someone suggests g() might be the problem, you are assuming they are
wrong and demanding an excess level of rigor from them. That's exactly what
Scott Alexander is warning against.

[edit: I'm sticking to epistemology. I'm not hijacking the thread to discuss
gender.]

~~~
aristus
Still going on that vein, eh? If you notice a global imbalance in CPU usage,
you should be totally focused on _understanding the cause_ , not inventing
excuses why g() is not the problem or why people investigating f() are wrong.

I frankly don't care whether it's a pipeline problem or a filter problem, or
both. I have more than a decade of performance work under my belt; I'm
familiar with the usual arguments. What makes me take notice is that so much
effort is put into arguing why there isn't a problem at all, or that people
who say there is a problem are somehow mistaken.

Seriously. Do you think that women are intellectually flawed? Yes or no? If
yes, I'd love to hear why. If no, what is your best guess why they are
underrepresented? No tricks, no shifting of levels. Why do you think it's so?
"I don't know, let's find out together" is a perfectly good answer.

~~~
glenra
> _Do you think that women are intellectually flawed? [...] If no, what is
> your best guess why they are underrepresented?_

My best guess is that relevant male math-type skills have a higher _variance_
than female math-type skills. That means that even if the _average_ ability is
about the same, there are more men than women at the very top of the skill
distribution and ALSO at the very bottom of the skill distribution. If you
look for math morons, you'll find more men; if you look for math geniuses,
you'll find more men. If you look for people with roughly average math
ability, you'll find more women. If the variance is even a LITTLE larger, it
can explain an arbitrarily large imbalance at the high end, and we have both
(some) statistical evidence that it IS larger and plausible biological reasons
to expect that it MIGHT be larger.

Women have two X chromosomes; men only have one. All the important stuff that
makes us human is on the X chromosome. So for the sake of intuition, let us
imagine that there were only THREE available genes for math ability, all
equally prevalent in the population, and any X chromosome you inherit includes
one (and only one!) of these three "math ability" genes. The genes are:

(1) "moron" (recessive)

(2) "genius" (recessive)

(3) "average" (dominant)

In that world, men would have a 1 in 3 chance of being a genius and women
would have a 1 in 9 chance of being a genius.

Similarly, men would have a 1 in 3 chance of being a moron and women would
have a 1 in 9 chance of being a moron.

In that world, having two X chromosomes makes women "more average" than men,
which means that if some career requires the "math genius" gene, that career
will be 75% male.

Does that all make sense?

Okay, now let's suppose that "genius" gene is a bit more rare. Let's say that
only 10% of X chromosomes have the "potential math genius" gene. In that case,
men would outnumber women 10-to-1 instead of 3-to-1 in professions requiring
it. The rarer that gene is, so long as that gene is either _recessive_ or to
some degree _averages out_ with its counterpart on the other X chromosome, it
will be hugely rarer that women fully express the properties that gene codes
for.

If this analysis is valid, it suggests that women are less likely to be at the
very TOP of math-related fields (or will have to work a lot harder to get
there) for essentially the same reason women are far less likely than men are
to be colorblind.

(relevant study:
[http://www.ncbi.nlm.nih.gov/pubmed/7604277](http://www.ncbi.nlm.nih.gov/pubmed/7604277)
)

------
incision
Personally, what concerns me most often in argument these days isn't any sort
of peer to peer inconsistency, but that argument often takes place in and is
hurt by the broadcast mesh of blog posts, responses to blog posts, comment
sections and tweets.

In isolation, any two peers are either going to fail to negotiate entirely or
converge in short order.

There's little hope for convergence of vocabulary or context much less rigor
among the chatter of a hundreds, thousands or millions of nodes trying to
dictate the protocol. More likely, you end up with a split brain where each
island establishes its own consensus totally apart from the other.

------
im3w1l
I think it is interesting to note that machine learning is succeeding where
the philosophers failed, in defining common sense notions like these.

------
true_religion
> they’re just out to steal cows.

What does this idiom mean? I've never heard of it before.

~~~
qianyilong
The reason you haven't heard of it is that it is a new phrase that he just
made up in the article to refer you back to the case of the philosopher
selectively applying logic so he can take his neighbors cow and justify it by
this being a new cow. It basically means that you are grasping at whatever
logic supports your pre-existing desire/belief rather than using logic to
determine your desire/beliefs.

