
Discrimination in the Age of Algorithms - martinlaz
https://arxiv.org/abs/1902.03731
======
nostromo
The paper's authors have the best of intentions, but to me race-aware
algorithms programed for social justice sounds completely dystopian.

If a black man and a white man rob a bank together, is it fair that they
receive the same sentence? Or is it fair that a sentencing algorithm give the
black man probation and send the white man to jail to correct for disparate
outcomes based on race?

~~~
obastani
The problem you mentioned is not the one that is being tackled. The real
challenge is that being blind to just race _isn 't enough_. For example, zip
code is often highly correlated with race, so using zip code to predict
whether an applicant would repay a loan may implicitly be discriminatory. We
need to know race to determine whether a given covariate is okay to use. But
we would usually not use race to predict outcomes (except possibly in a
limited number of affirmative action settings, such as college admissions).

~~~
whatshisface
Ok, help me reason though this. Let's say that native americans are poorer on
average and therefore less likely to repay a $1k loan. Therefore, if you give
out loans based on the probability that someone will be able to pay them back
you will be discriminating against native americans (not the individual native
americans with good credit scores, but you will be denying more native
americans on average). So, you shouldn't try to compute how likely it is for
someone to pay their loan back. I keep hearing things like, "you can't use
zipcodes as input because zipcodes are correlated with race and it is illegal
to discriminate on race." However, some races have better financial situations
on average, which means that financial situations correlate with race, and by
the same argument you shouldn't take finances into account when you review a
loan application.

Obviously that is absurd, but where does it go wrong?

~~~
kokokokoko
Nope. You are exactly correct. The way to approach removing bias in all
personal financial decision making, algorithmic or otherwise, is to only allow
it to be based on previous individual financial behavior.

The solution is pretty much that simple. I would argue that all correlative
models dealing with human beings will always be discriminatory.

Will loans become more expensive for many people? Sure. But that is the true
solution. Sometimes the right decision is simple but unconfortable. In my
opinion, this is one of those times.

~~~
whatshisface
> _is to only allow it to be based on previous individual financial behavior_

Your financial history includes your rent or mortgage payments, which are for
living at your zip code. Restriction to financial history is not all that much
of a restriction.

------
jondubois
As soon as you label people based on superficial personal traits that are
outside of their control and without their consent, that's discrimination.

If you start giving some people special treatment just because of their
ethnicity or gender, you're going to be opening up more cracks in the system
for other people to fall through - and it will hurt those people even more
because they will rightly feel that the whole system is working against them
personally; down to the core specifics of who they are (unfortunate
individuals who don't fit under any special label).

Modern anti-discrimination approaches are often racist or sexist. They should
not try to classify individuals into superficial groups, instead, they should
be decided on an individual case-by-case basis based on the individual's
history.

The root of injustice is simply bad luck. Ethnicity and gender are only
loosely correlated with bad luck but there is no causal relationship between
them (not so much anymore at least).

Anti-discrimination efforts should be aimed at averaging-out the effects of
bad luck in people's lives; so that means we need a way to quantify luck on an
individual basis. It's a difficult problem to solve, but it can't be solved by
making gross generalizations.

~~~
trowawee
> The root of injustice is simply bad luck. Ethnicity and gender are only
> loosely correlated with bad luck but there is no causal relationship between
> them (not so much anymore at least).

This is wildly ignorant and ahistorical. There are literally thousands of
books, articles, studies, movies, etc. disproving this. Literally just this
week a study was released showing that predominantly white school districts
collectively receive $23 billion more in funding than predominantly black
districts, and that: "For every student enrolled, the average nonwhite school
district receives $2,226 less than a white school district,".[1] That isn't
bad luck; that is a system designed to reinforce a system of racial
segregation, and it's just one example of thousands, all of which are a moment
of research away if you actually cared about this.

[1]: [https://www.npr.org/2019/02/26/696794821/why-white-school-
di...](https://www.npr.org/2019/02/26/696794821/why-white-school-districts-
have-so-much-more-
money?utm_source=twitter.com&utm_medium=social&utm_campaign=npr&utm_term=nprnews&utm_content=20190226)

~~~
DoofusOfDeath
The NPR story you site contains this quote: "As Rebecca Sibilia, founder and
CEO of EdBuild, explains, a school district's resources often come down to how
wealthy an area is and how much residents pay in taxes."

Based on that, any causal relationship between race and school funding is far
from simple.

> That isn't bad luck; that is a system designed to reinforce a system of
> racial segregation

Who you believe is designing the "system" for the purposes of segregation, and
what evidence do you have to support it? Asking because everybody I've met who
works in education seems to genuinely want all students to thrive.

~~~
DanBC
You can only say "The root of injustice is simply bad luck. Ethnicity and
gender are only loosely correlated with bad luck but there is no causal
relationship between them" if you ignore the years of segregation and red-
lining and other racist policies designed to keep the Black man down.

------
Cynddl
From Jon Kleinberg also, I highly watching his recommend his presentation [1]
on the inherent trade-offs when trying to achieve algorithmic fairness.

It's a little more technical than this paper, and presents very well why
removing discrimination in algorithmic decision making is a complex task.

[1]
[https://www.youtube.com/watch?v=4X3Z7FPwkA8](https://www.youtube.com/watch?v=4X3Z7FPwkA8)

~~~
nostromo
Note that this paper is arguing _for_ race-aware algorithms, not removing
discrimination.

If I'm understanding the authors correctly, they're saying that blind
algorithms don't have equal outcomes by race -- so they're reintroducing race
to the algorithms so they can adjust for disparate outcomes.

Basically they're arguing for affirmative action algorithms.

~~~
currymj
it is way more complicated than this. i know i underestimated how complicated
these issues can be at first. there are many different failure modes.

the race-blind models aren't necessarily really race-neutral; they could
implicitly have something like "affirmative action" in either direction, and
reintroducing an explicit race variable could be the only way to get rid of
it. or the opposite might be true! it takes real care to get this stuff right
whether notion of equality you aim for.

------
eponeponepon
This cuts both ways, of course - in some ways, I think we _want_ automated
systems to be able to discriminate. A good example is speed cameras - in days
past, a man speeding to get his wife to the maternity ward could be pulled
over and then hurriedly wished good luck, maybe even getting an escort. These
days, he gets a fine in the post three weeks later.

~~~
tedivm
How would an automated system actually deal with that scenario?

~~~
eponeponepon
In theory, the same way as a policeman - assessing the situation and the
context, and then making a decision not to issue the fine.

But in practice, I don't think it can, without having _total_ surveillance
capabilities - not to mention essentially sci-fi quantities of AI advancement.

~~~
gumby
In principle both systems permit this kind of correction: you could contest
the ticket by going to court.

