
Computer Scientists Find Bias in Algorithms - consciousbot
http://spectrum.ieee.org/tech-talk/computing/software/computer-scientists-find-bias-in-algorithms
======
gavazzy
First, some definitions:

Equality of opportunity: everyone takes the same test, outcome is dependent on
individual, even if the group containning that individual scores differently
than other groups

Equality of outcomes: everyone takes the same test, but the individuals scores
are weighted based on the group of which they are a member, so that
representation of groups is equivalent across all groups.

Further, we can separate bias in terms of intentional and unintentional, where
the former indicates direct manipulation of selection criteria in order to
change group outcomes to match the manipulator's desires. The latter occurs
when systematic bias affects the individual's scores, such as when a
particular group has less access to educational opportunities.

The article values equality of outcomes, by encouraging algorithms to take
into account the individual's group when performing selection. But in cases of
unintentional bias, this can mask problems. In the long term, it would be more
beneficial to ensure groups have equal access to education than to upwardly
adjust their scores.

In fact, this upward bias to adjust for perceived unintentional lowered
outcomes can in fact perpetuate the initial biases: the message is that a
particular group underperforms, and thus _needs_ extra help.

------
Houshalter
This is totally ridiculous. All it does is remove the useful data from the
dataset and makes it's predictions worse.

Imagine are using an algorithm to predict how likely someone is to pay back a
loan, or make an insurance claim, or perform as an employee, etc. Things like
their grades or education would correlate with that. but it might also
correlate with race or gender, so the algorithm would be labelled "racist" and
"sexist".

Even though it has nothing to do with racism whatsoever. A white person and a
black person with the same education would get exactly the same result. The
algorithm is totally fair and meritorious.

The true solution would be to condition on the race or sex. So the algorithm
predicts the probability they will be paid back _given_ that they are male or
female. Then in production you average the results of each possible sex and
race together.

So if women were really better drivers than men, they would still pay the same
rate. The algorithm would know that they are better drivers, but it would
treat everyone gender neutral. But it would still take into account education
or other variables.

Although it's not clear to me why you would want to do this. If men really are
more risky drivers than women, they should pay more. And they do. If they
really are less likely to pay back a loan, then they should have a higher
interest rate.

In any case, humans make decisions about stuff like this all the time and they
are way, way, worse and more biased. In most cases very simple statistics
vastly outperform the best humans:
[http://lesswrong.com/lw/3gv/statistical_prediction_rules_out...](http://lesswrong.com/lw/3gv/statistical_prediction_rules_outperform_expert/)

------
Malician
"If the test results show that the algorithm under question can distinguish
attributes—like whether a data point represents a male or female—then it’s
labeled biased. The other algorithm the researchers introduced tries to remedy
the bias by modifying the actual data set so that any selection algorithm
would deliver fair results. The algorithm does this by blurring attributes
that may be correlated to, say, race or gender."

So if you're testing construction workers, you identify bias, then you select
strength as the variable causing bias, and you adjust the strength values of
the dataset or force the algorithm to devalue strength?

Racism/sexism are real and we need new ways to fight them, but this is like a
right wing satire of affirmative action.

~~~
0xcde4c3db
I think the desirability of eliminating this kind of bias depends on the
context of the problem that the algorithm would be used to solve. Per your
example, suppose that strength is what an algorithm is _predicting_. We might
not want the algorithm to lowball the strength of a candidate just because
that candidate "looks like a woman" due to some other causally unrelated
correlation.

~~~
Malician
Sure, and I agree. But that's not what they're doing here. Paper is available
online:

[http://arxiv.org/pdf/1412.3756v3.pdf](http://arxiv.org/pdf/1412.3756v3.pdf)

------
current_call
The basis of the argument is that if decisions correlate with race, than the
cause of the decision is race. This is of course wrong. Correlation does not
imply causation and the algorithms cannot even have a concept of race or sex
unless that data is deliberately and explicitly placed into the data set.

A realistic complaint would be that these algorithms dehumanize people and
create a weird algorithm based bureaucracy that mimics The Trial.

Maybe someone is trying to conflate the two arguments so they can dismiss the
second one by pretending it's the first?

