
Forensic Pseudoscience (2015) - huihuiilly
http://bostonreview.net/books-ideas/nathan-robinson-forensic-pseudoscience-criminal-justice
======
todd8
I had a shocking experience with "expert" testimony in the case of a large
lawsuit I witnessed.

This expert, paid by the litigant, had written a few papers for low-quality
journals and taught at a class at a west-coast medical school on forensics.
His written claims supporting the lawsuit were provided to the defense and he
was questioned in a deposition.

The expert's conclusions in his report were based on bad probability. This was
several years ago so I don't remember all of the details, but one error was
the using 50% as a prior probability because the actual prior condition wasn't
known and could have been one of two possibilities, X or not X; 50% was picked
because there were two possibilities and disregarded the relative rarity of X
in the general population. Another error was really quite laughable, at one
point essentially arguing that P(A & B) = P(A) + P(B). All of this was hidden
by somewhat confusing language.

It really wasn't a laughing matter because the lawyers, which had asked me to
explain the report to them, couldn't understand the probability well and were
consequently unsuccessful in exposing the expert's faulty reasoning during his
deposition. Fortunately, the expert's so called report was not introduced at
trial and he wasn't called upon during the trial.

~~~
MrMember
Bad probability calculations show up a lot in the court room. A lot of
probability theory is counterintuitive so people can get away with making an
intuitive sounding argument that is fundamentally wrong. Two high profile
cases that I like to bring up are OJ Simpson's murder trial and Sally Clark's
trial.

I don't remember the exact numbers used in OJ Simpson's trial but the
rationalization used is more important than the numbers. His lawyer argued
that some very small percentage of men who beat their wives later go on to
murder them. Therefore, the fact that OJ beat his wife shouldn't factor into
his trial at all. The fundamental flaw in the argument is that there was no
question on whether or not his wife was dead. We knew she was dead, so the
correct statistic to look at would be, of women who were beaten by their
husbands and later murdered, how often was the husband the murderer? And the
answer was some large percentage, like more than 90%.

Sally Clark's case was even sadder. She was eventually released from prison
but committed suicide not long after. She was on trial for murder after her
second child died from SIDS, her first child having died the same way. The
prosecution brought in an "expert witness" to give the probability of one
woman having two children die from SIDS. All he did was take the incidence of
SIDS at the time, 1 in 8500, and square it, coming up with 1 in 75 million. No
other evidence of murder was found, she was convicted because of the
statistical argument. This argument has multiple flaws. It assumes that the
two incidences of SIDS in this case were independent events, which is not
necessarily the case. On top of that, even if you could assume they were
independent events, a 1 in 75 million event doesn't mean the event is
impossible. A person should not be convicted on that probability alone with no
other evidence.

------
imh
I wonder if whoever is supplying evidence via tests should be forced to
(blindly) apply the tests to people other than the defendant. If your test
says some random sample is more likely match than I am, it's very weak
evidence.

~~~
qrbLPHiKpiux
Forensics is the only field of study I can think of where double blind studies
aren’t routinely done.

~~~
sigstoat
this is nonsense.

there are certainly a number of forensic fields which are garbage.

other portions are fine, and are effectively analytical chemistry applied to
legal cases. are you aware of a lot of double blind studies in analytical
chemistry? what would that even mean?

~~~
kd0amg
How much of the conclusions drawn from these investigative techniques are
justified entirely by analytic chemistry, and how much goes beyond that? Using
an example from a related article, there's a substantial difference between
being able to say what's the chemical element composition of a particular
bullet and being able to identify the particular manufacturing facility and
batch it came from. The latter relies on a whole lot of assumptions about the
ammunition industry.

------
sigstoat
the US legal system relies on prosecution and defense both functioning. when
expert testimony comes into play, that means there should be prosecution and
defense experts. unfortunately public defenders are underfunded and frequently
inexperienced, and can't/don't get opposing experts to review evidence.

many of the state lab testing scandals would be discovered years earlier if
defense lawyers regularly had things retested.

------
dayofthedaleks
Cui bono?

~~~
empath75
There’s no incentive to convict the right person, if you’re a prosecutor or a
cop.

~~~
dsfyu404ed
Well, other than the fact that often the person who did it is often easier to
convict than someone who didn't.

~~~
michaelmrose
Are you entirely sure? Someone's race, ability to pay a lawyer, and prior
history can be huge factors in convicting them.

Convicting a poor black man who dropped a cigarette butt at the scene hours
prior to the crime may ultimately be far easier than finding the actual perp.

------
howard941
(2015) but still relevant

~~~
sctb
Updated, thanks!

------
jrochkind1
betteridge's law of headlines strikes again.

------
JoeAltmaier
Lots of strawman examples. Anything imperfect will have errors. I didn't see
much 'scientific' about the criticisms in the article. For instance, measures
of how often techniques are right, how often labs are correct.

~~~
michaelmrose
How do you propose a scientific study be done of the end result with a billion
confounding factors. Further the people on trial are heavily predisposed to be
actually guilty to start with supposing we aren't just picking random people
off the streets. They are the opposite of a random sample. Using nonsense to
"prove" that the actually guilty is still false even if the ultimate answer
was known by other means to be correct.

The logical thing to validate is the actual underlying science that is vastly
easier to tease out and test independently and scientifically if we haven't
even tried to disprove something it has limited merit and historically if its
proponents resist even trying to prove it it is almost certainly hokum.

Wherein the means to test and validate the means we are using to deprive
people of their freedom and sometimes their lives represents a fraction of the
budget of the criminal justice system we are guilty of malpractice as a
society.

This isn't something you can casually shrug off.

~~~
JoeAltmaier
Nor would you dismiss it without some kind of measure. The article was no
better than the worst of the practices it criticized?

~~~
michaelmrose
You don't need to disprove measures that nobody has proved in the first place.
Much as the burden of proof lies with the prosecution the burden of proof lies
with those who assert they have the wherewithal to prove the defendants guilt.

This assertion is entirely inline with proper scientific thinking.

