
Justice.exe: Bias in Algorithmic sentencing - ahakki
http://justiceexe.com/
======
Robelius
Forgive my ignorance since I don't have a CS degree, but don't all algorithms
used for selecting have a human behind the logic. So wouldn't an algorithm
always have some bias based on the weights given by a human. Kind of like
OKCupid saying you are a 75% match with someone. You may not be compatible
with that person, but whoever designed the algorithm declared it to be true
with their design.

So wouldn't any algorithm always have some biased in it?

Sorry if this comes off as a stupid question or is unclear.

~~~
untog
It's not a stupid question at all, and it cuts to the core of a lot of
problems with AI and algorithmic-based anything.

Any sufficiently complex programming project will end up reflecting _some_
assumptions and biases on the behalf of the programmer. As more programmers
contribute to a project, it won't reflect an individual's biases, but will
still reflect those programmers as a group.

~~~
gravypod
> Any sufficiently complex programming project will end up reflecting some
> assumptions and biases on the behalf of the programmer. As more programmers
> contribute to a project, it won't reflect an individual's biases, but will
> still reflect those programmers as a group.

Can you break this down more? It doesn't make sense to me. If you're writing a
machine learning application to take a dataset and match future inputs to past
results I don't see how these biases can sneak into the program.

Unless the programmers are changing the datasets then I don't see how this
makes sense.

~~~
woodruffw
In the case you presented, the biases are in the dataset itself.

Arrest statistics in the US are heavily skewed by race. If you were to take a
dataset of all US arrests between 1900 and 2000 and ask which populations are
most likely to "commit crimes" (i.e., be arrested), you'd get racially biased
model without recording _why_ it's biased (discrimination, enduring poverty,
minimum sentencing, &c).

~~~
mattnewton
Dead sibling comment asks a common questions I hear: why can't we just not
tell the algorithm race? And the answer is that, with enough data points, if
there is a statistical bias in the dataset, the algorithm will likely learn a
substitute for race in the other features. For the sake of argument, perhaps
an interaction between home address, annual income and model of car driven is
predictive of race, and race is predictive of recidivism in the dataset- then
the algorithm will learn this cross of features is predictive of recividism,
even though we would like all of them to ideally be irrelevant to sentencing.

~~~
gravypod
> home address, annual income and model of car driven is predictive

Why the hell would _those_ be factors? The factors of the case should be
things that actually matter in the case.

A rich person who murders someone should get the same sentence as a poor
person who murders someone.

A black person who murders someone should get the same sentence as a white
person who murders someone.

Adding those factors would be insane in the first place. If you're adding
crazy things like that you might as well add factors like "Can Juggle" and
"Can Burp the Alphabet" because things like that should have just as much to
do with sentencing as what kind of car you drive or where you live.

~~~
jcranmer
One example that's going to come up is "Has the person previously been
convicted of a crime?" That question is strongly correlated with race, and yet
is strongly arguable to be relevant.

Model of car is hard to say that it's useful for predictive value, but home
address (=which neighborhood did you grow up in, e.g., are you in Cabrini-
Green or are you Gold Coast?) and annual income (poor people are more likely
to commit crimes than rich people) definitely are going to be queried inputs.
Particularly if the question is things like "what should I set bail at?"

~~~
gravypod
> "Has the person previously been convicted of a crime?" That question is
> strongly correlated with race, and yet is strongly arguable to be relevant.

Past behavior is a good predictor of future behavior in my experience. That's
more than doubled when it's recurring behavior.

> home address (=which neighborhood did you grow up in, e.g., are you in
> Cabrini-Green or are you Gold Coast?) and annual income (poor people are
> more likely to commit crimes than rich people) definitely are going to be
> queried inputs

Why?

> Particularly if the question is things like "what should I set bail at?"

That's still the job of the judge and there is law at hand for how those
values are calculated. The computers won't be involved with that (I'd assume)
just sentencing based on previous case decisions.

~~~
pdkl95
> Past behavior is a good predictor of future behavior in my experience.

That question includes the past behavior of a lot more people than just the
person with a criminal conviction.

> recurring behavior

Institutional racism has recurred a lot in the justice system.

------
Houshalter
Humans are a thousand times worse than algorithms. Unattractive defendants get
prison sentences twice as long as attractive ones. Judges give significantly
harsher sentences just before lunch, when they are hungry. And of course the
classic biases against race and gender and other protected classes. I bet
there are many other biases that we haven't even discovered yet. Some studies
have shown that job interviews are worse than just looking at resumes.

Humans should never ever be in charge of sentencing decisions, or anything
else you want to be "fair". There's nothing remotely fair about human
judgement.

One particularly weird bias humans have, is we strongly distrust machines.
Psychologists have studied this and named it "Algorithm Aversion":
[http://opim.wharton.upenn.edu/risk/library/WPAF201410-Algort...](http://opim.wharton.upenn.edu/risk/library/WPAF201410-AlgorthimAversion-
Dietvorst-Simmons-Massey.pdf) Even when we know machines do better than
humans, we are much more distrustful of them. We can watch humans make
mistakes and an algorithm make many fewer mistakes, and still end up trusting
the human far more.

And it's weird, because in almost every domain that's been studied, algorithms
do vastly better than humans. For any statistical problem where there are a
few features available to predict the outcome, even simple linear regression
typically beats human experts. We suck at probabilistic and statistical
problems. Often the predictions of experts are barely better than chance. And
this has been known for a long time. The earliest reference I found was from
1928! Where an incredibly crude statistical model (hand trained!) beat a group
of prison psychologists, at predicting recidivism of convicts.

Even the sources that inspired this game have been shown to be fraudulent.
That is the propublica study they cite. See
[https://www.chrisstucchio.com/blog/2016/propublica_is_lying....](https://www.chrisstucchio.com/blog/2016/propublica_is_lying.html)
Propublica claimed their results were "almost statistically significant."
I.e., they not statistically significant. And why would it be so? Algorithms
have no particular reason to hate a minority group. They try to make the most
accurate predictions possible given the data.

~~~
dsacco
_> They try to make the most accurate predictions possible given the data._

So what you're saying is that a bias in the data could result in a biased
outcome.

~~~
magwa101
There will be bias in the data, but that can be looked at in one place, and
corrected once. At which point everyone gets the benefit. This means errors
are found fixed and "released" broadly and quickly. This is much better than
trying to disseminate training and information to judges who may or may not
actually absorb it. For example new information on bias could be encoded in an
algo but is possibly not teachable...because people are biased.

------
twiss
Small bit of feedback on the game: not being a law student, it took me a
minute to figure out what I was supposed to do. From the mention of "Risk
Assessment Program", I infer that I'm supposed to assume that everyone was
found guilty, and guesstimate whether they are likely to commit more offenses
in the future (and choose a sentence based on that). Is that correct?

------
tscs37
There has been some interesting discussion on this thread already.

However, what most seems to fail to see is that an algorithm compared to a
human justice system, could inherently be less biased.

Having no bias at all is hard but only having less bias is totally fine IMO.

When writing the algorithm it is of course impossible to not instill some bias
into the datasets, the code itself should be bias-free if written correctly.
The end result is a bias-free judgement algorithm based on biased data.

Is this worse than what we have now?

------
magwa101
How about the danger of HUMAN sentencing, talk about lack of training,
transparency and consistency.

------
Hydraulix989
One of the things that computers really have going for in the case of
"algorithmic sentencing" is that they are NOT biased; they are just machines
that repeatedly and reliably run an ordered set of instructions X, their
program, free of any human emotions that can override such logic.

Algorithmic sentencing, when implemented correctly (i.e. the X that it runs is
not biased), have the potential to be vastly more "fair" and "just" than any
human jury could be.

For example, from a widely circulated online video of a joint talk given by a
lawyer and a cop at a law school:

"the [Jury's] perception is that if you're sitting next to a defense attorney,
YOU have to prove you're innocent" [1]

Juries are comprised of people off the street who have no expertise in
determining things like innocence. The average person is subject to all sorts
of biases, most of which they aren't even consciously aware they have.

Knowing this, if I ever had to stand for trial in front of a jury of my peers
as a defendant, I'd be scared shitless, _especially_ if I was innocent.

As you can see from the linked video, machines actually have a pretty low bar
to live up to in order to outperform humans.

[1]
[https://www.youtube.com/watch?v=08fZQWjDVKE&feature=youtu.be...](https://www.youtube.com/watch?v=08fZQWjDVKE&feature=youtu.be&t=16m46s)

~~~
hackuser
People will write that software, select which software to purchase, write the
rules for its use, and implement them. Powerful people aren't going to sit
back and let those things happen in ways that they don't like, which are
threats to them, or which reduce their influence.

~~~
Hydraulix989
Then we need the software to be implemented as a decentralized smart contract.

~~~
twblalock
How would that help?

More specifically, how would implementing this as a smart contract prevent
bias from making its way into the algorithms or the learning datasets?

Remember that bias can be inserted accidentally, by well-meaning programmers
who don't realize that the data they are using is biased or has other
methodological problems.

~~~
Hydraulix989
To me, it just seems like the problem is solvable by having really clean data
(Maybe single data point has to be verified by a DNA test? There's plenty of
ideas here) -- much more solvable than taking a random group of (probably)
unqualified people off the street, calling them a "jury", and then having them
make potentially life-or-death decisions regarding the fate of their peers.
Right now, the existing system we have is so stacked in favor towards the
wealthy and Caucasian that I can't possibly see a well-intentioned algorithm
doing any _worse_ than what we have now (a study even successfully correlated
good looks with less-strict judicial outcomes; it's pretty easy to design an
algorithm that doesn't give an extremely lenient sentence to a Stanford
swimmer), but at least, it has a fighting chance of doing _better_.

Try reading anything at all about the jury selection process, and you'll be
shocked at how abysmal our existing system is.

People tend to dismiss machines as being cold, calculated, and emotionless,
but that's exactly what we WANT if we're aiming for equal, fair justice for
all.

~~~
twblalock
Clean data is basically impossible to get. You won't find very many experts
who agree what it would even mean for data about these topics to be clean.
There are tons of methodological disagreements, not to mention the fact that
even if we started collecting new "clean" data, the historical data could
never be used.

The law is a human construct, not a computer program. It is precisely because
of issues of bias, fairness, methodology, etc. that the law is complicated and
needs to be administered by people who can understand the complexities and use
their education and initiative to ensure fair outcomes.

It's not perfect, and it never will be. But the legal system was designed with
the assumption that judges and lawyers would be able to use discretion. To
apply rigid computer logic on top of a system that was intentionally designed
to be subject to human discretion would be a disaster.

~~~
Hydraulix989
I'm a bit more optimistic. For example, for all intents and purposes, I would
consider a positive DNA test result to be overwhelmingly damning proof of
guilt.

Indeed, although unpopular, technological advances in things like data logging
and DNA testing are able to establish both guilt and innocence in a much more
scientific manner than a judge crippled by such things as their current mood
(maybe they had an argument with their spouse before hearing the case and now
they're feeling particularly vindictive) and whether they "like" the defendant
as a person (maybe the defendant looks like their high school bully).

The Innocence Project has successfully exonerated many convicted "criminals"
solely on the basis of DNA evidence, some of whom have been behind bars for
many years. If that doesn't point to an extremely biased and flawed existing
judicial system, I don't know what does.

When I think of bias in the judicial system, I don't immediately point the
finger at machine learning algorithms and say "if we implemented these types
of systems, then we'd be biased" \-- instead, I look at peer juries and law
enforcement officers as the source of bias, things like "Stop and Frisk" or
"driving while black." You have very faulty systems in place like plea
bargaining which effectively nullify due process and encourage false
confessions in order to obtain a guaranteed lighter sentence. You have
prosecutors with their own illicit motives like appearing in public to be
"tough on crime" resulting in such atrocities as Aaron Swartz -- this very
concept of "making an example out of somebody" is a biased one.

Someone brought up the point that "bias can be inserted accidentally, by well-
meaning programmers" in an unintentional matter. On the other hand, right now,
instead of employing some well-meaning programmer who is trying their very
best to make sure things are fair, you have policies in place like racial
profiling which are INTENTIONALLY biased. Which is worse?

In my mind, ML has the potential of creating a bias-free judicial system more
than we could ever hope for by continuing to rely on fickle, highly error-
prone, and even outright malicious humans. People just need to get past the
fact that a "cold-hearted," emotionless computer is in the judge's chair
because that's EXACTLY what we want.

~~~
twblalock
> People just need to get past the fact that a "cold-hearted," emotionless
> computer is in the judge's chair because that's EXACTLY what we want.

That is not what I want. You should not want that either. If our current laws
were implemented strictly and impartially, without reference to mitigating
circumstances, the results would often be unfair, unjust, and morally wrong.

An emotionless computer would not consider any mitigating or unusual
circumstances unless it was programmed to do so in advance -- but not all of
the the circumstances that might come up in court are going to be obvious to
the programmers.

Even foreseeable mitigating circumstances would be very difficult to program
correctly, e.g. the difference between valid self-defense and manslaughter
depends heavily on the state of mind of the defendant. Good luck teaching a
computer how to deal with such a problem in a fair and just way, unless you
can manage to develop the first artificial general intelligence.

~~~
Hydraulix989
"Good luck teaching a computer how to deal with such a problem in a fair and
just way"

Good luck getting a HUMAN to deal with such a problem in a fair and just way!

~~~
twblalock
Humans do just that on a regular basis.

~~~
Hydraulix989
I don't know if I can believe that, given they also say that obviously-guilty
people with "affluenza" should get off scot-free from murder charges.

~~~
twblalock
So, for you, one bad outcome negates a preponderance of correct outcomes?

Even in the best legal system in the world, you'll be able to find a rapist or
murderer who got away with it. That's not a valid counterargument.

