
The False Promise of DNA Testing - sergeant3
http://www.theatlantic.com/magazine/archive/2016/06/a-reasonable-doubt/480747/?single_page=true
======
alexandercrohde
I don't think anybody is arguing that DNA testing could never be useful. What
this piece is exposing is that a system we trust to be 99% accurate (because
the science, if properly executed is) becomes tainted by improper procedures.

We need to determine processes that make bias impossible. For example, samples
should be sent without any markings, directions, or names. Simply to be
"compared." Periodically, an oversight group should send known matching or not
matching samples, and if an incorrect result is returned by the lab, the
analyst should be fired. Should multiple analysis failures happen under one
manager, than manager should be fired. This isn't rocket science.

~~~
Gibbon1
I'm always shocked that there is no auditing going on at all. If a lab isn't
blind audited periodically how can the 'evidence' be considered admissible?

~~~
fucking_tragedy
There are real consequences to ascertaining the accuracy of such evidence.
Cases get reopen and verdicts thrown out.

For years, prosecutors told jurors that DNA and hair tests were much more
accurate than they truly were, some cases claimed that they were 100%
accurate. Jurors then convicted possibly innocent people based on that lie.

The result was many convictions can now be overturned, prosecutors reputations
sullied and millions of dollars wasted.

The people that pay to keep these labs running have no interest in digging
that deep.

~~~
Gibbon1
I remember a friend made a good point once, there is arguing for truth like a
scientist, and there is arguing for truth like a lawyer.

------
bonniemuffin
Well, this is horrifying:

"In creating a DNA profile for the victim, Kim had typed three separate
samples, two from blood and another from saliva. The resulting DNA profiles,
which should have been identical, varied substantially. This alone was cause
for serious concern—if the tech couldn’t be trusted to get a consistent DNA
profile from a single person, how could she be expected to make sense of a
complex mixture like the one from the vaginal swab?"

~~~
danieltillett
Like any forensic test DNA testing is only as good as the operator. The poor
regulation and testing of these labs is certainly a huge worry.

------
new299
It's infuriating to read about DNA fingerprinting being used such critical
cases when DNA sequencing could be used to far more accurately and robustly
compare sample and targets identities.

Fingerprinting uses common relatively large changes in the genome as proxy for
an individuals identity. It uses these locations because that's what you could
do in the 1980s easily. It's possible to screw up both the experiments and
interpenetration relatively easily.

Technology has moved on, you can now sequence a complete genome for around
1000USD. Comparing complete genomes would unambiguously assign identity. You'd
effectively just be doing a string comparison over two very long, unique,
strings.

The only reason we still use fingerprinting is that forensics is highly
conservative. I'd guess it'll take at least another 10 years for them to move
to sequencing (if not longer). It is however infuriating.

~~~
pbhjpbhj
Whatabout identical twins, chimaera, ...

Also, in practice you'd be doing partial string matches presumably.

~~~
new299
One other point about twins. Twins while genetically similar, are not
identical [1]. I'd guess that the differences are still significant enough to
be able to assign identity through whole genome sequencing.

[1] [http://www.scientificamerican.com/article/identical-twins-
ge...](http://www.scientificamerican.com/article/identical-twins-genes-are-
not-identical/)

~~~
pbhjpbhj
Your citations says:

>"Geneticist Carl Bruder of the University of Alabama at Birmingham, and his
colleagues closely compared the genomes of 19 sets of adult identical twins.
In some cases, one twin's DNA differed from the other's at various points on
their genomes. At these sites of genetic divergence, one bore a different
number of copies of the same gene, a genetic state called copy number
variants." //

So, it's "in some cases" monozygotic twins differ by having copy number
variants [TIL! thanks]. So, not all cases (according to this citation) and so
the point still stands that "Comparing complete genomes would unambiguously
assign identity" appears to be false (albeit at a very low false match rate).

Interestingly the article says that copy number variants appear to arise as a
subject ages. Doesn't this mean that different cells in the body will have
different multiples of particular genes; so a mismatch wouldn't necessarily
mean that the DNA wasn't from the same twin?

------
superobserver
Title should read, "The False Promise of Flawed DNA Testing". It isn't
surprising to see prosecutors resort to underhanded methods in their zeal for
imprisoning others.

The basic science is sound. However, there are gaps in it. For instance, most
DNA testing techniques do not account for methylation patterns in DNA samples.
This is important for profiling epigenetic traits.

Ref.:
[http://www.atdbio.com/content/56/Epigenetics](http://www.atdbio.com/content/56/Epigenetics)

~~~
new299
DNA fingerprinting is a poor approach based on a 1980s technology.

When you're trying to determine identity I'm not sure that methylation has
much value.

------
marze
"A recent study in the journal Criminal Justice Ethics notes that in North
Carolina, state and local law-enforcement agencies operating crime labs are
compensated $600 for DNA analysis that results in a conviction"

This was by far the best and funniest part. What would happen if a suspect
offered to pay the same lab $600 if there wasn't a conviction?

------
readams
The article points to cases especially where mixed or degraded samples can be
interpreted incorrectly as the technician is forced to make a judgement call.
This actually seems amenable to statistical analysis to me however, so I'm
wondering why that doesn't happen here. You should still be able to assign
probabilities based on the evidence.

Perhaps they still present things like 60% or 40% chance of match to the jury
and they just hear "DNA match" and tune out the rest. I've said this before
but I'm beginning to suspect that along with a public defender there should be
a public forensic scientist assigned to defendants.

~~~
new299
It is to be frank crazy that we're talking about anything less that a 99.9%
match with today's technology.

We have platforms that for less that 1000USD can sequence (read) a complete
human genome. Each individual has millions of mutations which make them
unique. Each position is read with >99% accuracy. Just to make sure in general
each position is read >20 times.

For forensics it would be like comparing two very long strings for (very
nearly) exact identity.

The only reason we use techniques like DNA fingerprinting is because that's
what was feasible in the 1980s and forensics is an extremely conservative
discipline.

------
Puts
I've always wondered if there's some elements of the birthday paradox with DNA
testing?

~~~
Puts
Maybe I should clarify my thinking here. I mean that if a jury are told that
the DNA matches the prosecuted with 99% certainty, then most people would
interpret that as the person is guilty without doubt. But if you said that
theres a 99% statistical probability of two of the 57 persons in the same
court room where born on the same date, then I think most people would be very
doubtful, and even if you checked and it showed that indeed two people in this
room where actually born on the same day, people would see that a coincidence,
not statistical proof.

~~~
OneTwoFree
Assuming DNA matching has 99% certainity. Are you saying if I get DNA samples
from 57 different people and match each of the 57 samples in all possible
combination of pairs (so 57*56 tests), then there is 99% chance that I'll find
2 samples which will match? Someone should do this experiment. I think it's
very scary if this is true.

------
VikingCoder
It is NOT "false promise." (Or at least you don't yet KNOW if it will
eventually be "false.") It may not be working well NOW, which is unfortunate.

Let's not forget how much people (like Marvin Minsky) crapped all over neural
networks. And now they're beating professional Go players and doing Natural
Language Parsing better than anyone else.

~~~
mindcrime
_Let 's not forget how much people (like Marvin Minsky) crapped all over
neural networks_

AIUI, Minsky and Papert "crapped all over" perceptrons specifically and not
neural networks in general. It's often been said that the idea that "Minsky
said NN's don't work" was an erroneous interpretation by others in the field.

~~~
VikingCoder
He actively fought to keep funding from going to NNs. Actively. For years.

