
Are courts keeping “junk science” out of the courtroom? - EndXA
https://www.psychologicalscience.org/publications/psychological-assessment-in-legal-contexts-are-courts-keeping-junk-science-out-of-the-courtroom.html
======
geophile
The New Yorker had a really interesting article about arson forensics, a few
years ago. It interleaved the case of someone being executed for arson, with a
thorough debunking of the science behind his conviction.

And then fiber analysis has been debunked too.

And the list goes one: hysteria over child abuse combined with well-meaning
but incompetent interviewers creating false memories in supposed victims. The
problems with coerced confessions.

It's all quite terrifying.

~~~
djsumdog
There was a Citations Needed podcast about pseudoscience in the criminal
justice system:

[https://citationsneeded.libsyn.com/episode-94-the-goofy-
pseu...](https://citationsneeded.libsyn.com/episode-94-the-goofy-
pseudoscience-copaganda-of-tv-forensics)

It's an interesting podcast, but their interviewee sucked and she didn't
really cite any actual science; just hand waved away things.

I've found the studies they cite on fingerprints (your fingerprint could match
up with 1 in ~300 people; they're not as unique as once though -- the study
that's often cited is from the FBI) but I can't find anything on gun/bullet
print identification.

Supposedly you can't match a bullet to a gun, especially with more modern
firearms with precise milling. Yet all the sourced I've found should photos of
bullets with distinct patterns that seem to match a particular firearm.

~~~
manicdee
Then we find that all these photos are stock photos from the depths of FBI
labs and none of the photographs show an actual slug collected from a crime
scene.

------
tyingq
I heard an interesting one on a Dateline. Cops had some DNA they could not
match. So they did phenotyping and turned the info (hair color, eye color,
etc) into a sketch. And that was enough to get court orders for people that
matched the sketch to submit DNA samples.

There was a video of the police department announcing it, and even endorsing
the company (Parabon) in the announcement.

Maybe not complete junk science, but sketchy legal ground to me.

~~~
SQueeeeeL
It's sketchy legal ground. Basically they could say "everyone with blonde hair
and brown eyes must submit", and through other extralegal means have found out
the culprit had blonde hair and brown eyes. They manufactured a way to search
and arrest them.

~~~
rtkwe
My question is was this everyone over a large area or were they going to a
pool of people they already had some suspicion on. If they have a group of
suspects and are using this to narrow down who to actually have submit samples
it's more ok to me.

~~~
tyingq
From an existing pool.

------
alistairSH
It isn't news that the courts allow whatever junk science leads to
convictions.

Courts have allowed drug-sniffing dogs as evidence for ages. They're barely
more accurate than a coin toss and easily persuaded to give false "testimony"
by their handlers.

~~~
mjevans
A coin toss with little false negatives might be a good filter for a more
annoying or expensive test. I think that would be a fine use of the tool even
if I agree that as primary evidence it might be lacking.

~~~
jariel
Yes, that's just it, proportionality. Dogs might be good to sniff out
contraband or whatever to flag searches, but perhaps not as incontrovertible
evidence of something.

~~~
notahacker
How often do cases rely on dogs as 'incontrovertible evidence of something
though'. It's not like they testify!

If the dog's mentioned at all, surely normally it's background context to why
the person was searched and the drug found [and if the police are lying about
the finding, it's not the dog's fault!]

~~~
DanBC
In some places they have "fruit of the poisoned tree" \-- there are clear
rules for searches, and evidence obtained without using those means is not
admissible in court.

It feels weird that law enforcement get just avoid all of that by training
dogs to alert at a handler's signal.

~~~
alistairSH
I didn't elaborate too much in the first post, but this is the gist of it.

Any search that was enabled by a dog should be considered "fruit of the
poisoned tree". Between their lack of accuracy and a dog's bias towards its
handler, they should not ever serve as a basis for search or arrest.

When the justification for initial stop/search comes up in court, the dog's
hit is held up as incontrovertible proof that a search was justified. Courts
have upheld their use many times, all the way to SCOTUS IIRC.

This approach is far to prone to misuse by law enforcement. It's bad enough
they can pull you over without cause (most states have loosely defined
"irregular driving" laws), they can initiate a search of your car just as
baselessly.

~~~
jariel
""fruit of the poisoned tree". Between their lack of accuracy and a dog's bias
towards its handler, they should not ever serve as a basis for search or
arrest."

I think this is upside down ... the police are never going to have
'incontrovertible proof' of contraband before they do a full search _by
definition_. If they did need such 'total evidence' to search, they wouldn't
need to do the search at all ... they could just make the arrest.

And a dogs hint is not presented as evidence of a crime, merely reasonable
suspicion for a search, which is what we want the tool for.

The point is to provide some degree of a reasonable basis for suspicion, after
which more effort is used to determine the material outcome: either you have
1Kg of coke, or an illegal gun ... or you don't.

The entire point of the tools are to use something that demarcates reasonable
suspicion; dogs, and other things, are viable tools.

If you attempt to enter a country, and you are asked where you are staying but
can't provide an answer, or if you do not have a return flight - these are
indications of 'reasonable suspicion' of further questioning.

~~~
alistairSH
_The point is to provide some degree of a reasonable basis for suspicion_

SCOTUS ruling notwithstanding, I don't consider a dog a reasonable basis for
suspicion. They're easily manipulated - may as well just come right out and
let the police search whoever they want, because that's what dogs allow.

------
plopz
Related video of Last Week Tonight on different kinds of junk science:
[https://www.youtube.com/watch?v=ScmJvmzDcG0](https://www.youtube.com/watch?v=ScmJvmzDcG0)

There does seem to be some issues with how the court system has failed to keep
up with the modern day. Issues with scalability, accessibility and science
literacy are leading to a general distrust of the system.

------
tyingq
Aside from courts, the constant use of garbage like polygraphs by the police,
employers, and others is regrettable.

~~~
astura
Interestingly, in the US most private employers are not allowed to polygraph
their employees

[https://en.wikipedia.org/wiki/Employee_Polygraph_Protection_...](https://en.wikipedia.org/wiki/Employee_Polygraph_Protection_Act)

>However, the act does permit polygraph tests to be administered to certain
applicants for job with security firms (such as armored car, alarm, and guard
companies) and of pharmaceutical manufacturers, distributors, and dispensers.
The law does not cover federal, state, and local government agencies.

Since doesn't apply to any governments (local, state, federal) so so
government employees are subjected to polygraphs.

Interestingly, it can be argued that polygraphs may have their place in a
limited amount of employment situations as long as the lying/truthful results
are thrown out as the test may work as a "placebo" for the interviewee to
admit to serious skeletons in their closet.

[https://www.theverge.com/2017/10/23/16511732/border-
patrol-p...](https://www.theverge.com/2017/10/23/16511732/border-patrol-
polygraph-trump-boots-on-border)

>Polygraph results are of questionable scientific value, but the interviews
have an unexpected benefit apart from their dubious powers of detection: under
pressure from interviewers, applicants frequently admit to wrongdoing.

To be clear I personally lean towards not using polygraphs at all for
employment situations.

~~~
staticautomatic
"it can be argued that polygraphs may have their place in a limited amount of
employment situations as long as the lying/truthful results are thrown out as
the test may work as a "placebo" for the interviewee to admit to serious
skeletons in their closet."

If the test cannot distinguish between true and false, then any purported
"skeleton" is as likely to be false as it is to be true.

~~~
0xcde4c3db
The idea isn't that the machine is actually detecting anything here, it's that
the general atmosphere of the "test" and a sufficiently convincing performance
of "the machine says you're not being truthful" will get people to volunteer
information that they would otherwise have kept to themselves. It's a
variation on an old technique where an interrogator will claim to have
incriminating evidence or eyewitness testimony that they actually don't.

~~~
thrwaway69
Now I wait for someone to complain about companies abusing their hires through
this in another hn thread for hiring practices.

------
raxxorrax
That has probably never been the case for psychological evaluation. As the
article hints at, the assessments should be under strong scrutiny. Not only in
the courtroom for that matter.

While I do think it preferable to get an opinion about criminal
responsibility, it should be considered as an educated guess.

~~~
Mirioron
Are these the same psychological evaluations that psychiatrists do to
determine if somebody can be involuntarily committed? Because that sounds like
an even bigger problem than this being abused in the criminal justice system.
At least in the latter you tend to have rights afforded to you, whereas in the
case of the former in some places they can literally even bar you from
speaking to an attorney.

~~~
openasocket
There are two situations in the US in which a person can be committed against
their will. The first is if the person is considered a threat to themselves or
others. I don't believe there's a standard evaluation for that. But the threat
needs to be immediate and explicit. A homeless person wandering through a park
shouting he's going to kill the president isn't going to cut it. However, that
person could wind up arrested, which brings us to the second reason a person
could be involuntarily committed: being mentally unfit to stand trial. And for
that there are standardized evaluations. Keep in mind the bar for being
mentally fit to stand trial is pretty low. It basically means you understand
what a judge is, what a jury is, what the lawyers do, being able to
communicate with and understand your lawyer, etc. If you are found unfit to
stand trial, you will be involuntarily committed until such a time as you are
considered fit to stand trial.

~~~
Mirioron
> _If you are found unfit to stand trial, you will be involuntarily committed
> until such a time as you are considered fit to stand trial._

So, indefinite incarceration without being able to see an attorney or even a
judge? Any system will have cracks and people do fall through those.

~~~
openasocket
You can still see an attorney while committed, and you will be periodically
re-evaluated and that goes before a judge. As for cracks, I'm aware of cases
where a person has been put on a 72 hour hold inappropriately, but one bigger
issue is declaring people mentally fit for trial who really aren't. I know
institutions in Florida have been criticized for basically doing the bare
minimum to make people appear to be just barely mentally fit.

You are correct that people found mentally unfit to stand trial can remain
committed for years, particularly those with drug-resistant schizophrenia.
Tragically, for some of the most vulnerable members of society this is the
best case scenario. People with severe schizophrenia who can't function often
end up on the street, suffering hardships and victimized. There are parents
who's child is on the street, getting frostbite and being assaulted, who go to
any psychologist they can find to try to find someone to declare their child a
threat to themselves or others so they can get into treatment. But they rarely
are. Often they just have to hope their child commits some minor infraction
like shop lifting and wind up declared unfit to stand trial so they can get
the treatment they need.

~~~
Mirioron
Then there are people like Gustl Mollath:
[https://en.wikipedia.org/wiki/Gustl_Mollath](https://en.wikipedia.org/wiki/Gustl_Mollath)

You make it sound like this is only used for good and that nothing can ever go
wrong with this system. That's not true. While you do get reevaluated, you get
reevaluated by colleagues of the person (or the same person) who initially
committed you. Furthermore, you're forced to take mind altering substances at
their say-so and are imprisoned in a place, where any sign of fear or anxiety
is counted against you.

Also, in some places it's possible for the psychiatrist to determine that
contact with an attorney would "exacerbate your condition" and can bar you
from seeing one.

I would much rather be on the street and free than indefinitely locked up and
treated as poorly as people in mental institutions are.

~~~
openasocket
Like I said, I can only comment on the system in the US. I hadn't heard of the
person you mentioned, but it seems this was a case in Germany where he was
found not guilty of charges on the basis of insanity, and held in a
psychiatric facility. I'm not entirely sure if that could happen in a US
court. Getting a not guilty by insanity verdict is rare, and would have to be
made by the defense with the consent of the defendant.

There is absolutely room for abuse in the system, in the US and any other
country, I won't argue against that. And I'd love to here any proposed reforms
or ideas you might have about improving that. Requiring the mental health
evaluator and the facility treating you to be separate would reduce the issue
you mentioned about being re-evaluated by "colleagues of the person (or the
same person) who initially committed you" and I think that is currently the
case in the US in some states, though I'm not sure. Also requiring that
someone always has access to an attorney would be good (again, I'm fairly sure
that's true in the US). I also think significantly more funding should be made
available to mental health institutions, allowing people to voluntarily enter
for free or for a small rate, expansion of halfway-house type facilities and
programs, restructuring the administration of facilities (a big complaint from
my father is that bureaucrats with little education often had control over
facility policies rather than the PhDs who dedicated themselves to studying
the best way to treat mental illness)

------
openasocket
My parents are both clinical psychologists, and my father used to do some
evaluations for the courts. If I remember correctly, he would do evaluations
to see if a person is mentally fit to stand trial, and would recommend to the
judge if the person would be better served with psychological help than prison
time. Some context:

\- Being mentally fit to stand trial is an very low bar to pass. You just need
to understand what a judge is, what a jury is, etc. Generally people who don't
pass either have some sort of intellectual disability or severe schizophrenia.

\- The article lists a ton of different situations where psychologists are
involved in the legal system, and I don't see a breakdown on what the
psychologist is attempting to do. I know my Dad didn't use much more than an
IQ test and some basic questions to try and diagnose a couple of clinical
conditions. I'd expect some amount of pseudoscience if one side in a messy
divorce hires a psychologist to try and show the other side is a bad parent,
or the defense in a criminal case trying to show the defendant was in a
fragile emotional state and deserves a reduced sentence.

------
cmhnn
First of all something called "Psychological Science" already raises my
hackles. Science in general has become so corrupt and politically oriented it
makes killing elephants to vilify your technical rival's creation look
understated. We have so much useless "science" being funded and won by
professional grant hounds that garbage is inevitable.

The US has a legal system not a justice system. I suspect that is true of many
countries. Once you are caught up in it then you will realize it isn't
personal and it has nothing to do with the truth or justice. We have
incentivized "winning" of trials and have blindly gone along with the idea
that having both sides behave horribly ensures fairness because well..What
else can you do?

Might be time to step back and think if a bunch of premises are working out.
Does giving some wannabe tyrant a lifetime job really result in good decision
making because they are "free" of political influence? Is it cool to allow for
things like fine print and to allow lawyers to be immunized from many bad
behaviors because lawyers keep writing rules in such a way to keep lawyers
employed? Is it really cool that a prosecutor can lock someone up for 20 years
who was innocent but that the recourse is some lawsuit where everyone pays for
the settlement and they get to basically get away with "oops"? Is it cool a
defense lawyer can "win" a case where they know their client is a child
molesting cannibal?

Philosophy and big questions may seem quaint and are often the target of
people who claim they are some form of scientist but for some reason my non
scientist self and many average people seem to be able to suspect that the
legal system is a bit fucked up, paying someone to publish papers on the
implications of tyrannosaurus rex having created a vegan matriarchy might also
be fucked up and if you let those things near each other some really bad stuff
happens unless you get serious about fixing the core of problems both of them
have.

------
hackermailman
They still use blood splatter experts which when I heard it as a juror seemed
like pseudoscience, extracting all kinds of assumptions.

------
gumby
As fingerprints and “lie detectors” are still generally accepted it seems the
answer is “no”

~~~
zchrykng
Fingerprints are actually pretty accurate, though no perfect. Polygraphs are
generally not admissible it court so I don't know what you are talking about.

~~~
fenomas
When last I read up on fingerprinting it was pretty dodgy - the problem being
not that it's inaccurate, but that nobody _really_ knows how accurate it is.
Apparently the professional society that analysts all belong to forbids its
members from participating in studies, taking the position that it's "100%
accurate when done properly" (which is clearly meaningless).

It's probably 4-5 years since I read about this, it'd be nice if things have
changed.

~~~
phkahler
>> Apparently the professional society that analysts all belong to forbids its
members from participating in studies, taking the position that it's "100%
accurate when done properly"

Refusal to run a gauge R&R while insisting errors are due to the operator is a
sure sign of a problem. If it hasn't been done, that's one thing but refusing
to allow it is quite another.

~~~
AnthonyMouse
It's the same problem as it is with most anything. They want to use huge
databases, but to do that usefully you need an unrealistically low false
positive rate.

Even if the rate was actually really good -- something like 99.9% accuracy --
if you apply that to a database with a ten million people in it, you get ten
thousand false positives, which is crap.

If you pick one of the matches at random, they want to be able to claim that
it's a 99.9% chance that it's them, but it's really a 99.99% chance that it
isn't. Or worse, since it's possible the true perpetrator wasn't in the
database.

Even for things like DNA that theoretically can do better than 99.9%, you
can't get much better than that in practice because before you can, the error
rate starts to be dominated by human error rather than coincident matches.

------
ci5er
Of course they can't. Most Americans believe in some junk or another.

------
dontbenebby
Betteridge's law of headlines strikes again

> _Betteridge 's law of headlines is an adage that states: "Any headline that
> ends in a question mark can be answered by the word no". It is named after
> Ian Betteridge, a British technology journalist who wrote about it in 2009_

[https://en.wikipedia.org/wiki/Betteridge's_law_of_headlines](https://en.wikipedia.org/wiki/Betteridge's_law_of_headlines)

