
The Impact of “Blind” Auditions on Female Musicians (1997) - qiqing
http://www.nber.org/papers/w5903
======
cottonseed
Someone should build an job search/interview site that gives "blind"
interviews: scrubs candidates (application materials, resumes, etc.) of
gender, race, age, etc. You might not be able to blind the whole process, but
you could blind the preliminary steps. Even that would probably effect the
composition of the final selections, as it did here.

~~~
picks_at_nits
A similar thing has been done many times by submitting the same resumé with
male and females names, or with names statistically more likely to be white
and names statistically more likely to be a person of colour, with different
ages, and so forth.

Every time they run a test like that, they discover that who you appear to be
matters more than your objective experience.

~~~
swatow
I know these studies are common (probably because they are easy to conduct)
but they don't measure the intrinsic bias of the interviewer.

What they measure is a combination of the intrinsic bias of the interviewer,
and the statistical inference that the interviewer does based on race/gender
(statistical discrimination).

Because a resume is a noisy signal, it is still possible that race/gender
contains extra information even after you have seen the resume. For example,
suppose people tend to exaggerate, and this exaggeration introduces some
randomness. Given the signal "lead some impressive project", there is some
probability that the person didn't really lead the project. Now if the
probability of exaggerating is the same, but a Black person or woman was less
likely to have lead the project a priori, then even after observing the resume
that claims to have lead the project, the a posteriori probability of having
lead the project is lower for the Black person or woman.

~~~
mturmon
Responding to your highly inflammatory hypothetical:

You have gone from a nostrum about an entire population ("...a Black person or
woman was less likely to have lead the project a priori..."), which could have
included thousands-to-millions of people, to a statement about one particular
individual.

The grossest error of this way of thinking is that it is mixing a vague,
dubious, and unquantified signal (your a priori "knowledge") with a very high-
quality signal (a specific and verifiable statement made by a single person
about a single project).

If you're really proposing to do some kind of "Bayesian" weighting of these
two pieces of knowledge, you're trusting your machinery for assessment of
probabilities way too much. That a priori knowledge is junk compared to the
statement on the resume.

Or, to look at it the other way round: If you're so well-calibrated that
you're taking population-wide information into account, I shudder to think
what you must be doing with other side information like the font, page layout,
semicolon count, or paper composition. Lump it into the prior! What could
possibly go wrong?!

I must add that you're deploying a hyper-logical argument in a real-world
situation in what is honestly a stupid fashion. Nobody who does real-world
inference should operate this way.

~~~
swatow
We are talking about academic studies, not how I would personally act.

You are asserting that the signal from race/gender is very noisy and the
signal from the resume is very precise.

We can debate the precision of the signal from the resume, but race at least
is highly predictive of many objective qualities, e.g. it is highly correlated
with IQ. So what you call a _vague, dubious, and unquantified signal_ is
actually a highly informative signal.

~~~
mturmon
I notice you went right from race being correlated with IQ to race being
predictive of IQ. Unsurprising.

~~~
swatow
This is a discussion on probability and statistics, so I was using the
technical terms. In statistics if A and B are correlated, then A predicts B.
But you're just a liberal who assumes everyone who isn't is dumb. Fuck you.

~~~
Nutella4
Hey, there's another thing that's very, very predictable: That a person whose
argument is making bigoted remarks and claiming they're neutral statistical
results very quickly devolved into saying 'liberals are stupid!'

------
beloch
If you're responsible for choosing applicants to interview from a stack of
resumes, you can do something similar by simply having someone else strip the
names out before you see them.

While you probably are not consciously racist or sexist, simply consuming mass
media and living in the society we live in gives us aliefs (subconsciously
held beliefs that we consciously know are wrong) that can affect our choices
without our knowledge. It's substantially harder to remove your subconscious
from the equation when conducting interviews, but at least choosing the best
candidates to interview is a step in the right direction.

~~~
growupkids
Which is why I'm always puzzled as to why job applications ask for information
that has no bearing on the process of qualifying an applicant, such as age,
gender, nationality, race and so on. Unless you are specifically looking for
someone of particular age, gender, nationality and race why ask for this
information?

~~~
DanBC
In theory that information is used by the company to male sure they are not
discriminating against any groups. Ideally that information is on a seperate
sheet which is never seen by people in the recruitment process; it goes
directly to HR to use for their stats.

~~~
aptwebapps
OT, but that's a funny slip:

"... by the company to male sure ..."

------
teraflop
> Female musicians in the top five symphony orchestras in the United States
> were less than 5% of all players in 1970 but are 25% today.

The article was published in 1997; it would be interesting to see some more
recent numbers. I searched briefly and couldn't find anything definitive.

------
tomlock
This seems to suggest that when we know someone is a woman, we judge them more
harshly.

------
dmritard96
To me this kind of thing begs questions around affirmative action and other
methods of non-'absolute-performance' admittance to jobs/schools/etc. I
totally agree that for one person to get to the same 'absolute-performance'
point can mean that different people had to come a lot farther than others,
but this article implies that it doesn't(shouldn't?) matter. In college
admissions though, its never colorblind, socioeconimically bling etc. I'm
never really sure I know where I stand as on some level the absolute
performance for a job is what matters to share holders, other employees and
even end users - but as a society, the idea of whole groups of people being
left behind seems shitty.

~~~
ubernostrum
Affirmative action in college admissions serves a different purpose; it's
explicitly _not_ a pure merit-based evaluation, and really is poorly
understood.

A sort of basic understanding of how to implement it would be to assume you
have a points-based system for admission, which takes into account things like
previous grades, standardized test scores, etc., and normally has a minimum
threshold of _n_ points for admission.

One way to think of affirmative action is that it says "people from this
group, on average, will score _k_ points lower not because of a lack of
ability or intelligence, but because of a lack of opportunities earlier in
life due to social inequalities", and then asks questions about whether a
score that is within _k_ points of _n_ should be admitted.

The real implementations are more complicated, of course, but this is the
basic idea. The main problem with it is that starting to apply remedies for
inequality after someone has already lived through 18 years of it -- and 18
important, formative years at that -- is nowhere near enough.

~~~
noonespecial
The best way I've heard it put is that the admission process seeks to measure
the "velocity" of a potential student and not just position. If someone has
come from much further behind than another, they may be a better candidate for
the school despite being at a slightly lesser relative position at the time of
application.

~~~
im2w1l
And has it been evaluated in this regard?

~~~
noonespecial
Yeah. Almost certainly not. But the woman who said this to me was obviously a
true believer and at the time it touched my geek sensibilities far more than
the cynical vision of a pantone swatch of skin colors with hash marks behind
them required to maintain state funding.

Some things you just really want to believe.

------
tomlock
Also interestingly, this study perhaps shows that there isn't a bias towards
hiring women in this case for "political correctness". The screen prevents
this.

~~~
mwill
This study was published in 1997 though. I'm not suggesting there is a bias
problem for "political correctness" sake nowadays, but I think the people who
would argue that there is wouldn't accept a study from 18 years ago as a
counterargument.

It'd be interesting to see a similar study done in 2015, for this reason, and
also just general comparison over time.

~~~
tomlock
Sure! However, I feel like the grand irony of these kind of studies is that
the same kind of unconscious selection against orchestra members is probably
also occurring in evidence acceptance. It feels like people are overwhelmingly
critical towards evidence that women are treated unfairly in a way they aren't
to other evidence. But hey, that could _just_ be a feeling.

~~~
swatow
Isn't the exact opposite also possible? That there are all kinds of biases in
the world, but studies are being selected for (a) showing a bias against women
and (b) showing that women are objectively as good as or better than men?

~~~
tomlock
To put it another way, the following article was posted, upvoted, and
generated almost no discussion regarding its "methods" and the "evidence" it
presented:

[https://news.ycombinator.com/item?id=8909954](https://news.ycombinator.com/item?id=8909954)

Is it _possible_ that this is a result of there being a bias against the
evidence presented in pro-woman/anti-bias articles, while pro-man articles
receive little attention? Have we sufficiently offset that bias?

~~~
xentronium
It did generate some discussion before.

[https://hn.algolia.com/?query=Is%20There%20Anything%20Good%2...](https://hn.algolia.com/?query=Is%20There%20Anything%20Good%20About%20Men%3F&sort=byPopularity&prefix&page=0&dateRange=all&type=story)

------
pervycreeper
The original study does not appear to have had any control groups,
unfortunately. There are too many possible confounding factors present to be
able to attribute any of the difference definitively to bias.

