
The Dangers of DNA Testing - seibelj
https://www.nytimes.com/2018/09/21/opinion/the-dangers-of-dna-testing.html
======
bjt2n3904
This is one of my favorite pet peeves: a smart person comes out wearing a lab
coat[1] and holding a clipboard. They make a claim, and one of two things
happen.

Firstly, news reporters with little understanding of the subject matter try to
explain it to the masses, and the results are disastrous. (Not to mention, the
drive for academia to publish papers that prove the hypothesis.) Little needs
to be said here.

The second is that I frequently see my peers latch onto any PDF they can
download that supports their position. "Look, look! The data shows I'm right!
And this man with a PhD supports my conclusion! (Of course I haven't _read_
the report, but that's not important. What _is_ important is my pet issue!)"
Prosecutors using sloppy lab work to get convictions and computer algorithms
deciding bail bonds are just end results.

We need to put the brakes on the empiricism train. It's useful, but only to a
certain point. Reason eventually needs to take over. If your argument lives
and dies by the next PDF or Machine Learning model, you need to find a new
argument, or bolster your argument with reason.

1 - [https://xkcd.com/699](https://xkcd.com/699)

(Meta argument: I'm reading a report about a paper someone published, and
latching onto the report because it supports my conclusion that people will
believe anything a "scientist" says! I'm victim to my own problems!)

~~~
dwaltrip
> We need to put the brakes on the empiricism train. It's useful, but only to
> a certain point.

We need to foster a deeper understanding of how empiricism actually works,
what its limitations are, and the difficult and often lengthy process through
which varying degrees of certainty are obtained and adjusted over time.

I also think it is important to spread the value of grounded skepticism
(including of one's own knowledge) and the power of freely admitting mistakes.

Obviously, none of this is easy. But it is worthwhile and I think small
improvements will have a large impact through ripple effects over time.

We shouldn't throw the baby out with the bathwater, in regards to empiricism.

~~~
karmelapple
Few people who haven’t done a variety of masters or PhD level research really
understand how there are only degrees of “knowing” something.

Teaching the scientific method early on, and really getting to the heart of
what it means to stay unsure about things, is something the education system
could do a better job of, but it’s no small task.

------
emodendroket
Since Sessions ended an inquiry into obvious junk science being used in
courtrooms (which itself seemed to be moving pretty slowly given the scale of
the problem), my prediction is that nobody will do anything whatsoever about
this finding.

~~~
amelius
Every defending lawyer will be waving this news article over their head ...

~~~
MBCook
Does it matter? If it’s accepted science then courts are loathe to just start
throwing it out even when there is very good evidence it’s junk.

------
dnafan123
Link to open access publication:
[https://www.fsigenetics.com/article/S1872-4973(18)30248-5/fu...](https://www.fsigenetics.com/article/S1872-4973\(18\)30248-5/fulltext)

Link to more detailed article: [https://www.forensicmag.com/news/2018/08/nist-
publishes-land...](https://www.forensicmag.com/news/2018/08/nist-publishes-
landmark-mix13-dna-study)

~~~
AlexCoventry
I can't understunderstand why the OP article didn't at least cite the paper.

I guess the quote works as a kind of content-based addressing, though.

------
YeGoblynQueenne
>> The good news is that there are methods to reanalyze old DNA mixture data
using computer programs that can help analysts correct errors, without any new
lab testing. In fact, one lesson from the study is that while only seven of
the 108 labs in the study properly excluded the innocent profile, one of them
used such a program (TrueAllele by Cybergenetics).

>> (...)

>> In fact, we have shown that this is possible. Working with Cybergenetics
analysts and Innocence Network organizations in four states, our Boise State
University laboratory has re-examined a few select cases and already persuaded
courts to overturn a conviction in New Mexico, two in Indiana and two in
Montana. We have also helped identify a new suspect in a 23-year-old murder.

This leaves a bad taste. Have I just read an advert for a proprietary piece of
software, masquerading as an article?

------
vedtopkar
It's shocking to me that something so critical to the criminal justice system
is not more tightly regulated nor monitored continuously for
sensitivity/specificity.

~~~
MBCook
Given courts have ruled drug dogs don’t need to be tested for accuracy or to
make sure they’re not influenced by their handlers this isn’t very surprising.

Seems like the more I learn about the system the more of a scam it seems.

Speaking of which Serial season 3 just started which covers this kind of
stuff, more in the small than a grand ‘Thing X is the problem we want solved
that will make things much better’.

~~~
ptaipale
I don't quite follow. Why would drug dogs need to be tested for accuracy? Is
the indication of a dog accepted somewhere as evidence?

I have been assuming that dogs give false positives every now and then, and
there's a search for some actual evidence based on that indication, which
everyone knows can be a false positive.

~~~
vkou
When the dog's accuracy can't be tested, then a search based on the dog giving
an indication of "There are drugs here" is no different then a search based on
a magic 8-ball giving an indication of "There are drugs here".

 _Every_ time this technique leads to a search that is used to secure a
conviction, it erodes your rights, as a non-criminal, against arbitrary search
and seizure.

~~~
ptaipale
There's a difference in the way people see this here in Europe, because here a
"probable cause" is not needed for searching for drugs. Particularly at the
border controls, everyone is subject to searches.

(Same applies for e.g. DUI testing on the roads; the police has the right to
take alcohol test of any driver, without anyone looking like they're driving
badly.)

Any arbitrary seizure, if only based on dog indication, is of course a
horrible miscarriage of justice.

~~~
vkou
Borders are civil liberty free zones, by design. As are warzones, places that
have declared martial law, and prisons.

DUI testing is different. In many jurisdictions you do not have to submit to
it, if you are fine with losing your driving privileges - which is not a
criminal penalty.

I'm not super pleased about it, but there is typically a clear line - some
voluntary behavior voids your liberties. Involuntary behavior generally should
not.

------
heyyyouu
There are many, many examples of DNA testing being unreliable because of HOW
it is done (for example:
[https://www.theatlantic.com/magazine/archive/2016/06/a-reaso...](https://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-
doubt/480747/) ) and yet the general public still sees it as infallible.

It's extremely important for these kinds of studies to get more publicity, for
the courts to start to understand the limitations, and most importantly for
the labs that do the tests to start to be held accountable. The standards from
a scientific standpoint are truly deplorable.

------
exhilaration
I know this doesn't help the folks in prison without any resources, but during
a trial, can't a defendant ask for any DNA test to be rerun by a private lab
at their expense?

~~~
dnbgfher
Even if that is an option, and the results come back favorable to the
defendant, they still don't have a great situation. Now there will be two
experts coming in and saying different things to the jury. The details that
might explain those differences are likely beyond what can be adequately
explained to a jury during a trial.

But the defendant themselves paid one of the experts, so...

~~~
oldandtired
Decades ago, I served on a jury where we had an expert witness for defence and
an expert witness for prosecution. We, the jury, made a decision to ignore
both expert witnesses as they each had a different opinion, please note the
word "opinion".

I do not know with what regard, under US jurisdictions, the evidence provided
by "expert" witnesses is treated, but at the time, we were told by the
presiding judge to treat it as opinion by an "expert" and not as direct
evidence.

------
sundvor
The article gets the timeline of Jackson's case completely wrong:

[https://www.innocenceproject.org/cases/dwayne-
jackson/](https://www.innocenceproject.org/cases/dwayne-jackson/)

This more detailed account states the exoneration happened _years_ after the
plea bargain release; the eff up discovery did not cause him to be released.

So the real impact on this poor man's life was thus significantly higher.
Imagine trying to get anything worthwhile happen in life with a criminal
conviction...

------
cheeko1234
If we can't trust witness testimonies, lie-detectors, and now DNA results,
what are we supposed to use?

~~~
Obi_Juan_Kenobi
It should be made clear that _most_ DNA evidence is quite reliable, provided
basic standard procedures are employed. I.e. as long as you don't switch
samples or otherwise mess up, the results are very robust.

The particular issue here is the mixed samples. The basic approach of DNA
fingerprinting is to look at a suite of variable markers which, taken
together, form a unique set of alleles that can positively identify an
individual. It's easy to see how a mixed sample can pose a problem: if you
have two contributors and 20 loci analyzed, that's over a million permutations
to consider, assuming all loci are different for those samples. Making an
affirmative match in this situation is much more difficult! Not only must your
statistics be much stronger (to offset the permutation complexity), but you
must make far more assumptions about the sample than before. Namely, you must
estimate the number of contributors to - and the relative proportions of - a
sample via various heuristics to arrive at a statistical metric of match
reliability.

It's kinda amazing that it can be done at all, but for smaller numbers of
contributors, it's not that bad. The classic example is a sample that contains
DNA from both the perpetrator and the victim; such 2-contributor situations
seem to be well understood at this point.

The main issue for reliability is when there are a large number of
contributors, and particularly when very sensitive assays are used. Increasing
sensitivity means you have to rely more upon amplification, and the
exponential nature of PCR makes it very easy for minor contributions to be
out-competed.

Basically, not all DNA evidence is equal, and it's important to distinguish
between more and less reliable methods. Unfortunately the vagaries of legal
precedent are often have an outsize influence on what is accepted in court.
The apparent reluctance by the NIST to report their findings is really what is
most concerning here, not the tests _per se_.

~~~
lutorm
Even if the actual labwork is flawless, there are still the issues of DNA
contamination where your DNA is spread around on objects and persons you
touch. And this problem just gets worse the more sensitive the detection
methods are.

------
spsrich
"It is uncomfortable to read the study’s authors praising labs for their
careful work when they get things right, but offering sophomoric excuses for
them when they get things wrong."

But not surprising.... they want to continue to work in the field, after all !

------
lucio
Isn't clickbait?

"The Dangers of testing DNA _mixtures_ " should be the title

------
anoncoward111
And justice for all. Technology can be horrifically negative when in the wrong
hands.

------
heyyyouu
Some articles are so important that they don't belong behind paywalls. I
believe this is one of them. The NYT's ombudsman should help develop a policy
to determine when that should occur.

~~~
Pinckney
Isn't that why we have scihub?

