
Machine Bias: Man Is to Computer Programmer as Woman Is to Homemaker? - acoravos
http://www.fatml.org/schedule/2016/presentation/man-computer-programmer-woman-homemaker
======
backpropaganda
If I were training a classifier to predict whether a sentence is talking about
household activities v/s not, wouldn't the occurrence of man/woman in the
sentence be a _good_ feature? Today, woman do perform household activities
more (whether we like it or not), and wouldn't it make sense to _use_ that
piece of information when performing some predictive analysis?

The technical sense of "bias" arises when the train and test distributions
differ. Obviously if you train with a dataset of text from a foreign country's
news and then apply it on an American context, the difference in the data
distributions will introduce bias, but why do we need a social twist to this
already well-functioning term? If the same classifier is trained and evaluated
in India (with its sexist roles, say), then there's no (technical) _bias_ and
I don't see why it's a bad application.

~~~
praxulus
>wouldn't it make sense to use that piece of information when performing some
predictive analysis?

No, because eventually your system will graduate from predicting the results
of society's bias to reinforcing society's bias. That is a bad thing.

~~~
backpropaganda
Can you give an example of a situation where an ML application would be
reinforcing a problematic bias but still have good performance metrics? My
point is that a wrongly-applied ML application would suffer in just plain
accuracy. For instance, a Automatic Carrier Counsellor might give "homemaker"
as a suggested career choice to women, but then before we start calling it
biased, it would already be _wrong_. If the same algorithm had dug deeper, it
would have learn that the said woman would be a great programmer.

~~~
praxulus
Recidivism prediction systems will usually tell you that black people are more
likely to get arrested/convicted again. They do so accurately, but also result
in longer sentences for black people.

[https://arxiv.org/abs/1610.07524](https://arxiv.org/abs/1610.07524)

~~~
xupybd
Yeah but doesn't that have more to do with the way the predictions are used?

It seems to me to be a stupid thing to do. This person seems more likely to
get convicted again, lock 'em up longer. Instead of asking why is this person
more likely to get convicted again? Can we prevent this in a redemptive non
punitive way?

It's really useful to have that prediction/data but how you use it is more
important

~~~
ZenPsycho
the problem is a layperson doesn't necessarily know what a prediction
necessarily means without a deep understanding of how the system is making its
predictions, let alone how to apply it.

worse is that since the prediction is coming from _computer_ that lends the
prediction an air of authority another article called "bias laundering". the
general belief is that computers are objective and cannot have bias, which in
a sense is true, but people don't tend to think a step further about the
problems and biases in the people who programmed the computer.

so that is definitely a thing usually missing from these discussions is that
the people using these systems generally don't know how they work, and believe
they predict or imply things that they don't

------
vtange
This is the tug-o-war of influencer v. influencee. A machine that just tells-
it-as-it-is might hold an advantage over one that willingly ignores some data
to promote a different view of the world.

Personally, I see more danger in people trying to make machines that
evangelize their own biases to the world than machines being molded by the
existing social assumptions of society, given that we expect machines to
perform most of the work/control most of the resources in the future.

------
mkrum
If you are going to "debias" your model, what is the point of even training
the model to handle these issues in the first place? Not surprisingly, human
language can be biased. If you train a model on human language it will not
magically transcend those biases. The problem is that people have this
expectation that ML is going to lead to these perfect decision makers.

Machine Learning creates models that reflect the data, not the truth.

------
reader5000
In the sjw-religion, why is "homemaker" considered inferior to "computer
programmer"? One of the oldest and most important human occupations versus
hunched over at a desk slaving for a salary until being outsourced to a bot in
5 years? I've never understood the default sjw/"feminism" assumptions that
anything feminine is "bad".

~~~
kingbirdy
No one ever became a billionaire by homemaking

~~~
kgwgk
[https://en.m.wikipedia.org/wiki/List_of_most_expensive_divor...](https://en.m.wikipedia.org/wiki/List_of_most_expensive_divorces)

~~~
AstralStorm
On the other hand, most of those women were already rich and powerful.
(including a more lucrative profession than housekeeping)

