
Machine learning in the judicial system is a bad idea - yosoyubik
https://palladiummag.com/2019/03/29/machine-learning-in-the-judicial-system-is-mostly-just-hype/
======
323454
In so far as the rule of law means ensuring that actions have predictable
consequences, I view it as imperative that we automate as much of the legal
system as is technically possible (not saying we're there yet). In the US,
federal law is growing increasingly complex, not to mention the added
uncertainty introduced by judicial activism (judges reinterpreting old laws,
ostensibly to make them fit modern society). As a result, the legal system is
completely backlogged and unpredictable, requiring people to expend a lot of
money retaining top lawyers who know how to game the system. If machine
learning can help make the system more predictable, it could, on balance,
improve the rule of law and the spread of justice, even if in a small fraction
of cases a decision is made that differs from what a judge would have decided.

~~~
wilde
Today’s ML does not produce predictable outputs. Introducing it into the
justice system would be worse than humans for the values you’ve outlined.

------
skywhopper
Bad headline. The author discusses why it would be a terrible idea to use
inherently flawed ML approaches to establishing statistical models of criminal
risk, and how opposed such a system is to the fundamental principles of the US
justice system.

But that doesn’t mean the talk about using ML to cut human empathy out of more
and more of the justice system is just “hype”. There’s no substance to the
techniques proposed, that’s true. They are more dangerously biased than the
humans they are meant to replace. But that doesn’t mean that these systems
aren’t going to be purchased and implemented if we don’t push back harder
against the general AI hype that pervades VC tech culture in general.

ML can recommend a movie to watch or a restaurant nearby, and when it fails
terribly and reinforces biases and misunderstandings like it always does, the
cost to society is minimal. But when you put people’s lives in its hands, the
consequences of the inevitable failures are far more damaging.

~~~
Barrin92
>ML can recommend a movie to watch or a restaurant nearby, and when it fails
terribly and reinforces biases and misunderstandings like it always does, the
cost to society is minimal

I'd go further and say we should pay a lot of attention to this too. The
subtle ways in which recommendation engines change what cultural goods we
consume, including journalism, might very well have some pretty significant
effects.

But yes, to essentially beta test rudimentary statistical models in actual
court rooms or police stations were people's legal rights are at stake is
pretty much insane.

------
SeeDave
I can only speak about San Francisco Pretrial Services, but they use the PSA
Tool by the Arnold Foundation
[https://www.psapretrial.org/about/factors](https://www.psapretrial.org/about/factors)

On paper, a ML approach to calculating risk of recidivism can make sense if
you consider it as a silicon-based approximation of the criminal brain in
making totally rational self-serving responses.

One caveat: even if you have a perfect ML system... never forget the 'human
factor' when using these tools. A single fat-finger by a clerk in San
Francisco lead to the release of a defendant who went on to commit murder in
2017. [https://abc7news.com/details-of-mistaken-release-of-sf-
twin-...](https://abc7news.com/details-of-mistaken-release-of-sf-twin-peaks-
murder-suspect-revealed/2312248/)

------
debacle
Machine learning is mostly just hype.

~~~
jaabe
We take it fairly serious in the public sector in Denmark. We’re doing our own
POCs and we’re inviting partners to build things with us. It’s very much a
hit’n’miss field.

It works with recognition. It helps us sort through millions of page files
looking for missing documents, both faster and with higher hit rate than using
people. It helps us determine how much water is in drone photographed land
areas based on the colours of the vegetation. Stuff like that.

Where it absolutely doesn’t work is for prediction and analysis.

Well, it’s not that it doesn’t work in BI as such. I kinda does, but we’ve
been working with analysis and BI for 25-50 years. IBM wanted to sell us
Watson analytics for BI as an example. We let them run their magic on some of
our non-sensitive datasets and they came up with a range of interesting BI
models and metrics. Every one of them were far, far inferior to our current
human build BI setup, however, and a subscription to them would cost us around
three full time analysts.

Prediction is much worse. The probability just isn’t there to use it for
anything that isn’t as harmless as advertising. Worse than that, almost every
model turns out biased.

So I guess you can say that I agree with you. It’s mostly just hype, but those
parts that aren’t hype are incredibly useful.

~~~
soraki_soladead
I'm not going to say that ML is a panacea or always perfect but its also not
one "thing". It's a complex field with a lot of required knowledge and ways to
set things up.

IBM is largely to blame for that misunderstanding since they brand all of
their offerings "Watson" and describe it as if it's one system; so if their
system doesn't work, maybe none will. In reality, it's dozens of pre-
implemented algorithms with varying degrees of quality. Importantly, how the
data interacts with those algorithms requires domain expertise for the data
and algorithms so it can be hit-or-mess if that expertise is not properly
applied.

------
pqdbr
Too broad of a statement. Title should have used 'Criminal System' instead.
Afterall, criminal trials are just a small part of the Judicial System.

Civil cases could benefit enormously from ML, helping judges with rulings
suggestions based on jurisprudence.

~~~
rayiner
> Civil cases could benefit enormously from ML, helping judges with rulings
> suggestions based on jurisprudence.

What makes you think “ML” would even be able to do that?

------
carapace
This seems to me to be a good article on the subject. It covers the bases
soberly.

> The legitimacy of the judicial system is based on its status as a well-
> tested, publicly examinable, and heavily scrutinized tradition of practice.
> A shift to an unaccountable, opaque software system throws that basis away.

To me this is the fundamental problem with ML poisoning judicial systems: For
better or for worse, we are abdicating our responsibility. Justice ain't easy.
Automation should free up our time for more important (less automatable) tasks
like justice and politics.

------
bryanrasmussen
Since machine learning allows you to codify the bias of the system maybe the
business to be built on top of judicial system is something that helps
individual citizens instead of companies - use cases:

1\. Tell you the chance you or members of your family will have a criminal
case. Early warning system - do I want to leave this area?

2\. Tell you what your chances are of conviction/time you will have to serve
if convicted once you have a case.

Once it gets into tax law it might be really interesting.

~~~
pessimizer
I'd love my demographic pre-crime report, somebody should get on this.

~~~
bryanrasmussen
It would be good if you could do stuff like give it an itinerary - I am going
fishing this weekend with my awful in-laws - and it could give you sort of a
list of things you might end up being charged with to look out for.

------
Causality1
Are there any cases of machine learning solidly outperforming other options in
real-world applications? Is there anything it does better than a person? You
can point at something like evolutionary algorithms and say "this antenna is
better for this use than anything designed by hand" but can you do that with
machine learning?

~~~
soraki_soladead
Evolutionary algorithms are a type of machine learning, so yes. Machine
learned vision, speech, NLP, etc. outperform heuristic methods and those are
just the recent advances.

A random forest can easily outperform a hand-made decision tree—and this is
the important part—if it is set up correctly and you use a sufficient quantity
of high quality data. Linear models have also been used for decades because
they work.

