Hacker News new | past | comments | ask | show | jobs | submit login
Twins get different results when they put 5 ancestry DNA kits to the test (cbc.ca)
574 points by Gaessaki 3 months ago | hide | past | web | favorite | 292 comments



None of the differences, either split by company or by twin seem egregious or even particularly unsatisfactory. They cite that AncestryDNA measured each of their DNA to be 99.6% similar, and really, that seems like the error rate you would about expect for the Ancestry results they got. They got the exact same regions and only a percentage point or two off in each region's share.

It's not like they're fully sequencing every DNA sample they get.

> AncestryDNA found the twins have predominantly Eastern European ancestry (38 per cent for Carly and 39 per cent for Charlsie). But the results from MyHeritage trace the majority of their ancestry to the Balkans (60.6 per cent for Carly and 60.7 per cent for Charlsie).

This part of the article especially seems like hair splitting considering the Balkans and Eastern Europe tend to have a lot of overlap. In fact, "Balkans" in particular is an extremely ambiguous linguistic term and can mean so many different things to so many people.


What's unsatisfactory is the way these companies present these results which implies a level of accuracy and precision which is unwarranted.

"She also has French and German ancestry (2.6 per cent) that her sister doesn't share."

I assume from this that 23andMe is reporting the results to this level of accuracy (a tenth of a percent). For the sister who got 2.6% they are strongly implying a certainty that she has a small amount of French and German ancestry rather than what actually seems to be the case which is a small chance of some French and German ancestry.

Since it seems their results are quite lacking in both precision and accuracy they should do a better job of reporting them and advertising them in a way that makes that clear.


I can only speak for myself, but when I got one of these tests 6 years ago, I never got such an impression. They've updated my results several times since and I never feel hoodwinked as if I had been previously deceived; I'm always just curious to see what the latest estimates are.

> I assume from this that 23andMe is reporting the results to this level of accuracy (a tenth of a percent). For the sister who got 2.6% they are strongly implying a certainty that she has a small amount of French and German ancestry rather than what actually seems to be the case which is a small chance of some French and German ancestry.

You picked out the single biggest discrepancy they report in this article, and yet even that doesn't seem to indicate an actual inconsistency to me. I would tend to interpret those results to mean that for one twin, they felt sufficiently confident to call a portion of their DNA "French and German", while in the other it fell below that confidence threshold so it got thrown into the "Broadly European" share.

This process already pretty much precludes objective perfection just based on what they're reporting. What does it mean for your ancestry to come 2.6% come from France? During what time period? What if it's on the border in one of those regions that switched hands several times? These are subjective decisions they have to make but it doesn't invalidate the whole test because occasionally you have enough evidence to call portion of your DNA "French and German" and sometimes you have to fallback to just calling it "Broadly European".


> This process already pretty much precludes objective perfection just based on what they're reporting. What does it mean for your ancestry to come 2.6% come from France? During what time period? What if it's on the border in one of those regions that switched hands several times? These are subjective decisions

I think that’s the whole point of the complaint here. These companies pretend to have found overly precise results (which I’m sure are explained away to nothing in the fine print) and yet are subjective and lean heavily on guesswork.


23andMe told me that I was a quarter Italian. Seemed to have mixed that up with what I actually am: 50% middle eastern. For a while I wondered if there was some funny business in the family, but then my sister got the "right" percentages. This seems like a pretty huge error to me.


I know it might can be assumed from the context of the post and thread but are you talking about your twin sister or not? Just pointing out that the chance for funny business (in your family or at your birth or at these companies) is unknown but not necessarily zero.


Well, Italian and ME populations were basically the same thing for more than a thousand years (basically until the Islamic Conquest promoted piracy on the Mediterranean, making it fundamentally unsafe). So it might well be that you are carrying genes that most of the Italian population also carries, and viceversa.


Is it possible one of you has a father that's 100% middle Eastern, and the other has a father 50% italian and 50% middle Eastern?


I would argue that this is why significant digits count.

You can't say something is 2.1374cm long when your error is +/-1cm. You can say it is 2cm long though.


Actually, you can. It's just a custom to do differently.

But if you estimate your first moment is 2.1374 and your second moment is 1, I want to know just that, because it is the numbers I'll plug in my equations. I will decide to round if and when necessary


You're implying that by rounding before handing off the numbers, valuable information is lost. Actually, the reverse is true - the rounding itself contains valuable information (how many digits are actually relevant) which is a hidden property of the process that obtained them. Unless you provide it through a side channel, failing to round hides this information, for no benefit at all (because the digits you provide are known to be meaningless).

The only time this wouldn't be true is if you mistrust the significant figure calculation of the source, and believe you have sufficient insight to do better.


Yes, I do. In general, I stick to simple things. Enough samples and everything is normal, god bless the law of large numbers.

If the precision of your estimation is not a direct function of the standard deviation, but is a "hidden property of the process that obtained" it, we have much bigger problems that losing "valuable information"


> If the precision of your estimation is not a direct function of the standard deviation, but is a "hidden property of the process that obtained" it

I think you're confusing different types of error. There is error between measurements and an inherent error to the device you use to measure. There's also a difference between precision and accuracy.

Standard deviation is the difference in multiple measurements. For example if you measure something 10 times to be 51mm, then your standard deviation is 0.

But that doesn't mean you have no error.

The "property of the process that obtained it" is not hidden. A simple case is a ruler. You have lines on the ruler that tell you certain intervals. If the smallest interval on your ruler is 1mm, then all your calculations can be made to +/- 1mm (that is, up to 30.5cm on a standard 12in ruler). There is nothing hidden about this. All that is being said here is that your measuring device is not perfect.

So using the two errors, we have a measurement of 51mm +/- 1mm (or frequently in a short hand you'd just say 51mm). It would in fact be deceptive to say that your measurement was 51.0mm, because that implies that you have more precision than you actually have (implying that you have on the order of +/- 0.1mm precision).


Assuming we are talking about software, error between measurements is a direct function of the device you use to measure, which is itself close to perfect

Even if we go to the example you give, the measurement should be done n times, each reporting the exact result found like 51.0 51.9 51.95 etc. Even if the decimals are outside the smallest interval of your ruler: take enough of them and you can get closer to the actual length which may be 51.55345 and that you would never have been able to measure anyway without a caliper

The best thing is you can even do that by resampling old measurements (a process called bootstrapping)

So yes, if you remove the tenth of millimeters, you lose information.

What's wrong is not the number, but that custom makes people think 51.0 means 51.0 +- 0.01 or anything else while it was never said like this.


Even ignoring the part where you are measuring past the precision of the measuring tool, you're still wrong. Making multiple measurements and then averaging them does not give you more precision just because you get a non integer (in this example) number. It does not bestow you with more precision.

Why you do multiple measurements is because as humans we are imprecise. Any engineer, wood worker, whatever knows the saying "measure twice, cut once". In your example, the thing you are measuring could well be exactly 52.0000...cm.

If you don't believe me I seriously want you to ask ANY physicist. They can even be an undergraduate (assuming they aren't a freshman) and they should know this. We even use this to figure out what we should spend money on. We can process these errors and determine which measuring device needs more certainty and buy that new device.

This is WHY we have very precise devices. With your method, we could theoretically get measurements to nanometer levels. I can tell you, I would much rather spend 15 minutes making a hundred measurements than spending thousands of dollars on a laser and equipment needed to make precise measurements down to the nm level.

To sum up:

> take enough of them and you can get closer to the actual length which may be 51.55345

NO! This is just dead wrong. It'd be right if you said 51 or 52.

> that you would never have been able to measure anyway without a caliper

No! This is why we have calipers!

> if you remove the tenth of millimeters, you lose information.

Not if you didn't have that information in the first place.

>What's wrong is not the number

Yes it is.

I can only assume: 1) You are trolling, 2) You are really thick headed, and/or 3) You've never taken a physics class. I'm not saying anything here that isn't easily verifiable. I have others backing me up. So if you have no interest in learning, then there is no point for me to continue.


How are you getting a measurement of 51.95mm from a ruler with 1mm ticks? If your ruler is capable of outputting that, and if those digits are actually meaningful (even if they do have error bars) then obviously yes they are definitionally significant digits and you shouldn't round them away.


Good point. Maybe not 51.95, but 51.9 mm you can say if you eyeball the measure to be quite close to the mm mark, but not exactly there.

Repeat enough time and you can interpolate 51.9, which you wouldn't have been if you had thrown away the precise measurement even if the precision is within your measurement errors (of 1 mm here)


Your smallest tick is 1mm, not 0.1mm. You can't measure to 51.9mm. So your measurement is 52mm.


Back in high school in physics we would get minus points if we indicated a too high precision in the numbers we used for calculations, it was considered plain wrong to say 2.232cm if you actually only were able to measure that it's roughly 2cm.


And that's good, because you'll get the wrong answer if you used the number with too many points (nit picking: more decimal points does not mean higher precision).

In fact, this is part of why you'll see physicists do all their reductions with variables and plug in numbers at the very end. This ensures that you doing get (what we could call) floating point errors. You don't have extra numbers hanging around (from real numbers like 1/9 or pi). There are also other benefits to doing this.


I'm not sure why anyone thinks significant digits in any way informs error rate, and it's a little hyperbolic to suggest 4 significant digits with a whole number error rate, when they did 1 significant digit with a ~3% to ?% error rate.


>I'm not sure why anyone thinks significant digits in any way informs error rate

In the majority of technical fields the standard is to round the result so that the significant digits do correspond to error rate.

https://www.ruf.rice.edu/~bioslabs/tools/data_analysis/error...


I'd expect that in academia for the most part. But this is a consumer product, so they aren't going to follow technical fields, and my hunch is that's for marketing reasons (it looks more authentic).

But even then, rounding to the error rate is a standard practice, but it certainly does mean it's wrong or even unethical, it's just pointless not to. Unless of course your motivation is driven by marketing/sales.


> it looks more authentic

One might call that deception


> This part of the article especially seems like hair splitting considering the Balkans and Eastern Europe tend to have a lot of overlap. In fact, "Balkans" in particular is an extremely ambiguous linguistic term and can mean so many different things to so many people.

We don't have a good way to name the origins of populations. We use approximates based on nationalities which is probably a really, really bad proxy. Take France, where there is probably not a single part with the same kind of gene pool (germanic invasions from the North, gallic roots in places that were untouched, italian tribes from the south and I could go on) so "French & German" as a split means absolutely NOTHING.


It doesn't mean _nothing_ -- it's just prone to the same type of problems and edge cases affecting pretty much every attempt at categorizing in a very complex system.


To me it means _nothing_ when the end-class you use is very ambiguous. A classifier should be as specific/differentiated as possible or else, it's useless.


You mentioned in another comment that haplogroups would be too foreign a classification for you, but that would likely be the most specific/differentiated classifier.

If somebody says they're a "quarter French" -- putting aside whether that's interesting or not -- do you believe that's meaningless? Do you demand them to specify whether it was one of they have Gallic roots or if it was one of the Germanic tribes that invaded from the north 1500 years ago or something else altogether?

I think the point of this test is to try to capture the spirit of what Americans have done for a long time, which is to try to describe their heritage in fractions using nationality of a distant (but not too distant past) as a rough guide.

And in the same way that your genome might prove more complicated than naming fractions by nation, somebody might say "My grandfather came to the USA from France... but he was part of a second generation immigrant community from a Greek-speaking part of Sicily who emigrated en masse to Alsace, which was located in Germany, not France at the time".

So maybe the DNA test will call that French or German or Italian or Sicilian or Greek or whatever and that might oversimplify it and the test doesn't work as well as somebody whose ancestors stayed in one location and was part of a largely homogeneous population, but that doesn't render the test meaningless. Part of the excitement people derive is that it can sometimes illuminate your ancestral history beyond what is well known. Using the previous example, somebody might not be aware of the complexity of their grandfather's heritage, and might be interested to understand that part of the family has roots in Sicily and not just France.


> We don't have a good way to name the origins of populations.

https://en.m.wikipedia.org/wiki/Haplogroup ?


Interesting. But it's far from being a well known denomination (layman-wise).


Disclaimer: I'm not an expert on the subject.

I remember watching PBS DNA documentaries[1] where it's mentioned in one of the episodes that DNA-wise there's only 1% difference between any two humans.

I looked for the same details today on genomenewsnetwork[2], it has following line

> we are all 99.9 percent the same, DNA-wise. (By contrast, we are only about 99 percent the same as our closest relatives, chimpanzees.)

It seems like even an error of 1% would be actually be same as missing the whole difference between two people.

  [1] https://www.youtube.com/watch?v=apOP8MMedqE
  [2] http://www.genomenewsnetwork.org/resources/whats_a_genome/Chp4_1.shtml


A lot of genomic tests only sequence the single base pairs that happen to contribute to that 0.01 percent variability. The article cites that only a subset of the genome was sequenced and that subset isn't chosen at random but rather highly targeted to get at variance in populations


These genomics services don't look at the parts of the genome that remain the same for everyone...


> In fact, "Balkans" in particular is an extremely ambiguous linguistic term and can mean so many different things to so many people.

Not really, Balkan countries are well-defined and have been so for over a century now. It's basically European countries that were under the Ottoman Empire:

Greece, Albania, North Macedonia, Bulgaria, Romania, Serbia, Kosovo, Montenegro, Croatia, Bosnia. One could also throw Slovenia in there but historically speaking, they have always been pretty different from other Balkan-folk.


"There is not universal agreement on the region’s components." [1]

"It can be difficult to define exactly which countries are included in the Balkan States. It is a name that has both geographic and political definitions, with some of the countries crossing what scholars consider the 'boundaries' of the Balkans." [2]

Almost the entire wikipedia article is dedicated to explaining the various definitions of it and how it is a problematic term. [3]

[1] https://www.britannica.com/place/Balkans

[2] https://www.thoughtco.com/where-are-the-balkan-states-407024...

[3] https://en.wikipedia.org/wiki/Balkans


I'm from Bosnia, which is in the Balkans by every definition of the term. As for the rest of our neighbors, it's a mixed bag.

According to certain definitions, only the coast of Croatia is in the Balkans (without Istria peninsula). Vojvodina (the northern region of Serbia) should definitely not be in the Balkans. Slovenia definitely not. Romania? I can't tell without looking up the definitions.

See the problem?


Just for the sake of accuracy - Croatia was never "under" the Ottoman Empire. That was Herzegovina - so technically today's Bosnia.

Also not sure what you mean by Slovenia being historically different - slavs, similar language, part of austro-hungarian empire, part of Yugoslavia, etc etc


You don't need twins for this test. A single person can just sign up twice for these services and get different results, even if they make a single sample and split it between the two vials.

I thought it was pretty well known that these services are estimates only to a large degree? These services only take samples, they don't sequence the entire genome. Of course there will be errors when you have to extrapolate from a sample.

Even for me personally, my results changed when my parents did it and linked up to me, because their results were able to flow into mine.


I havent looked into getting a DNA test at all. But I would expect it to be accurate and repeatable.


You expect it to be accurate and repeatable because you haven't looked into getting one.


Unlike what you might think, this is not a sequencing based method, but something based on dna sticking to small parts of a chip coated with complimentary sequences. This method is super error prone, more importantly the error rate is dependent on who does it and with what level of care. So yes it's not going to be repeatable!


It's true that these companies are not offering "sequencing" per se but rather are genotyping commonly polymorphic SNPs using an array or other genotyping method.

These SNP arrays, however, are exceedingly accurate and reproducible in their calls. A common medium-density SNP array used for human genetics is the Illumina CytoSNP-850K [1], which routinely gets >99.5% accuracy on calls. This is at least as good, or better than, whole genome sequencing.

The challenge in ancestry assignment is not determining what is in the sample, but rather in the interpretation and assignment of ancestral haplotypes.

[1] https://www.illumina.com/content/dam/illumina-marketing/docu...


Since the test results for the twins in the article were 99.6% identical, that seems to match what you say is the usual accuracy of SNP arrays. However, I don't think that can be called exceedingly accurate and reproducible if you consider the intended application.

>99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers.


If the practical difference is 38% vs 39% "Eastern European", who cares?


If you feel that way about 99.5% accuracy, I'd advise that there are a number of consumer products you absolutely shouldn't buy, including: kitchen scales (small ones), measuring cups, store brand spirit levels, etc.

You can get considerably better accuracy, you just have to be willing to fork over $1500 rather than $60. How much are you willing to pay to know if your 31% German vs 30%?


The difference is that it's not 99.5% accuracy for the entire experiment but for each of the millions of spots, which means there will be hundreds of thousands of spots that are not accurate in each chip. Not an apples to apples comparison.


">99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers."

Or potentially put people in jail based on these results...


I am inferring that the problem of matching a particular individual's results with haplotypes is a problem in optimization (finding a best fit) with many variables that is actually solved by maximization (finding a good fit) without doing a comprehensive search of all possible solutions. The space of potential results may be very large -- very many haplotypes from which to select and a percentage to be assigned to each, from 0 to 100, and a perfect result cannot be expected, as ancestors within a given haplotype are nowhere near as similar as identical twins. It is very common, even typical, for algorithms that attempt to find good solutions to such problems to include some randomization (making some guesses based on pseudo-random numbers) followed by more directed attempts to successively improve the guesses previously made. Because some steps of the search are a random walk, different results from the same inputs are common for several iterations of the same algorithm.


DNA microarrays do have limitations vs. RNAseq, but they are accurate and repeatable when used properly. https://en.wikipedia.org/wiki/DNA_microarray Calling them 'super error prone' is simply inaccurate.

There was a good 5-10 years where microarrays were the go-to method for genomic investigation. Since RNA-seq has supplanted it, there has been no revolution in overturning those old results. At best, RNA-seq reveals what microarrays cannot: novel sequences, specific sequences that have difficulty hybridizing, etc.

The variation between these services is due primarily to how the data is interpreted and presented to customers, not the reads themselves.


Note that these companies are not doing DNA microarrays in the sense you're describing (which is for quantifying gene expression by mRNA level). RNA-seq is (now) arguably better for this purpose as well (better dynamic range, primarily), but again this is not what any of the ancestry or personalized genetics companies are doing.

In this case "SNP microarray" is a "digital" (or binary) readout of the genomic DNA sequence; specifically the array has attached to it sequences that match both the "reference allele" (e.g., a "G") and the "alternate allele" (e.g., "A" or some other base). Note that across the world's population, most sites in the human genome are "bi-allelic" meaning that there's really only two variants at each site, though of course it is possible to have tri-allelic or even fully polymorphic (A, T, C, or G) sites. For the purposes of ancestry, bi-allelic identification of commonly polymorphic sites is adequate, though the more sites and the rarer those sites are in world populations, the more specific you can be re: ancestry. This is why the gold standard for ancestry and haplotyping is whole genome sequencing-- you simply get far more informative sites to map to other populations.


You do not offer any citation as to why this, the chip based genotyping, leads to misinterpretation and the article does not suggest this is the reason either. Without getting into it, the article says

>An entire DNA sample is made up of about three billion parts, but companies that provide ancestry tests look at about 700,000 of those to spot genetic differences.

>According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.

>Still, none of the five companies provided the same ancestry breakdown for the twins.


It's fair that a cheap test might not be super accurate, but then why are they stating figures like "13%" if the margin of error is at least 10 percentage points? That's just plain misleading.

When my doctor asks how tall I am, if I'm 6'1", then 6'1" is a good answer. 6' (or even better: about 6') can also be an acceptable answer, as it's an approximation. I wouldn't say I'm 6'8", and then try to rationalize it by saying my tape measure is cheap, and anyway it's still right to about 10%.

Reporting what you don't know can be just as important as reporting what you do know. It doesn't matter how good your laboratory method is if you can't report results in a way that doesn't mislead customers.


Ok this is new to me.

So these people getting results and finding out they aren't related to their parents, are the results suspect?

Ditto the results about genes associated with dementia or heart disease?

I'm aware of the companies having disclaimers, I thought that was on the basis of them not really know what the genes did, not that the test itself was rubbish.


He is just wrong, the twins in this case got the same "results" as far as the genotyping is concerned. The differences were in the analysis provided.


That would explain the differences between the companies, not between the twins, no?

Is it a case of them adding too many decimal places to the percentages? Making statistics out of noise?


he is dramatically overstating the error rate


But surely you wouldn’t expect it to be perfectly accurate and repeatable. It’s pretty well known that no medical test I can think of is perfect.


Pretty well known by who? The general population? I don't think so.

And there's a difference between "not perfect" and "doesn't provide the same result twice".


"Not perfect" is the hand-waving way of excusing the imperfections and proceeding as though it were nearly perfect

"doesn't provide the same results twice" is the honest acknowledgement that the concept of perfection is itself what is flawed here because it creates false expectations of what science can give us in this arena


No no no, "science" can definitely deliver repeatable and accurate DNA sequencing, this much is beyond question. But that's expensive, and these companies sacrifice accuracy/repeatability to make it cheap.


The accuracy of these tests at these prices is a falsehood that science cannot support

The accuracy of the broad conclusions drawn from population wide samples about any single individual’s ethno-historical origins is a falsehood that science cannot ethically or logically support


"Not perfect" is when it fails a small number of times.

Not getting the same result twice is pretty much 100% failure :-)


Really? Taking a pregnancy test more than once and getting a second opinion about a medical diagnosis are certainly both well established in public consciousness.


Maybe so but the kind of genotyping test that we are talking about is vastly more accurate than an over-the-counter pregnancy test, both in the mind of the public and in actual fact.


Yeah, I think most people would be surprised to learn that the same blood tests done by different labs can sometimes show 10% difference in values or even more. This could be the difference between a doctor declaring someone "within the normal range" or having a health problem.


I tend to agree, and since people are convicted and incarcerated on "DNA evidence," this seems an appalling and horrific indictment on the fairness of our court systems.


Comparing two DNA samples for identification of a culprit is totally different from determining the percentage of differing heritages that make up your DNA.

Frankly I am skeptical that they could even ascertain any semblance of your heritage but I'm definitely a layperson.


I had a one off blood pressure tests a few years ago that was outrageous 140 over something and completely unusual for me (which previously were like 110). The nurse was like, lets see what happens with a bigger cuff, and the reading from the automated machine was suddenly like 120 over something, and she was like, shrug you should watch it.

So I picked up a mid-range automatic bp cuff, and started to measure it more regularly. The numbers were all over the place and completely unrepeatable unless I maintained exactly the same position/breathing/etc. I can literally swing my BP by 30+ mmHg, by re-positioning my body, drinking a cup of coffee, etc, all while officially still maintaining proper technique for measuring it. And yes, moving up a cuff size also tends to remove a ~5mmHg from my measurement.

I could write a book about how to get any BP you want out of the machine. Its the same with weight, I can shift my weight +-10% by eating a bowl of salty popcorn drinking a few 64oz cups of ice tea, then eating a large meal. Then I can go for a 3 hour bike ride, use the toilet and drop it all back off.

So, its sort of surprises me the fine grained aspect of DR's always measuring your BP/Weight, neither of which really mean anything until they get into the extreem ranges, and in the weight case you can get a better feel for it just by looking at someone...


Where I trained, this is why we don’t diagnose someone as hypertensive unless there are three separate elevated blood pressure measurements on three different clinic visits.

We would schedule people for Nurse only BP checks to help establish the pattern.

This statistical anbiguity is why the medical community as a whole is often sceptical of the Direct to consumer DNA testing and SNP analysis. They report data in a convincing way which is statistically misleading and in no way plainly reveals what is within measurement error.


DNA tests are used in murder trials, I would expect them to be damn near perfectly repeatable.

Reading other replies there seems to be a cheaper less accurate tech used for these home testing kits.


DNA fingerprinting is so different from whole-genome characterization that it's hardly worth comparing them.

DNA fingerprinting involves:

* Sequence amplification of polymorphic sites by a panel of well-characterized primers, many of which cross-check each other by amplifying over the same locus.

* These amplicons are then characterized by size via restriction digest and simple electrophoresis. No sequencing is performed, only the length of the DNA fragments is characterized.

* Due to population-level variation of short-tandem repeats, analysis of ~20 loci can uniquely identify an individual with a desired level of statistical confidence. The kind of data used to confirm a match in e.g. a murder trial will be absurdly robust.

DNA fingerprinting is limited by the quality of sample collection and handling (i.e. don't mix up or contaminate samples), but not the accuracy of the method.


Your confusion is because you think there is one "DNA test". That isn't the case. You can sequence a genome, but that costs on the order of $1k so companies like 23andme don't offer that. They use simpler methods.

Even sequencing a genome is not perfect - the current most common method works by cutting the genome into pieces, sequencing those and then reassembling them. A pretty obvious limitation is that you can't detect repeated sections.

I'm not sure what method the police use but it definitely isn't the same as what 23andme do.


> DNA tests are used in murder trials, I would expect them to be damn near perfectly repeatable.

So are other forensic methods, like fingerprint[1] and hair[2] analysis. Turns out that forensic evidence isn't very scientific nor accurate.

[1] http://theconversation.com/fingerprinting-to-solve-crimes-no...

[2] https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-9...


> DNA tests are used in murder trials, I would expect them to be damn near perfectly repeatable.,

So were the results of FBI hair and fiber analysis, and look how that turned out.

https://www.washingtonpost.com/local/crime/fbi-overstated-fo...


The justice system deeply misunderstands both science and statistics.


It doesn't misunderstand them. It knows exactly what it's doing. It relies on judges and juries misunderstanding science and statistics in order to get results favorable to the prosecutors.

Basically all forensic science has turned out to be crap or misapplication of good science except for DNA which a) exonerates a heck of a lot of people and b) was a concept that matured in biology before being adopted by forensic scientists.


Even if the tech was the same, couldn't they just run the less accurate test several times to increase the confidence?


That will reduce the random error, yes. But there's still the concern of systematic error, which has to be dealt with separately. If you're taking a biased sample to begin with, it doesn't matter how many times you repeat the test; the bias will still be there.


But wouldn't systematic error affect everyone's results in the same way, meaning it couldn't result in any particular pair of people becoming more or less likely to be matched as the same person?


But it's comparing it to the corpus of known DNA in different location, which is changing and being updated even by the fact of people submitting more DNA samples from different locations.


Depends what you're getting and how much you're spending on it.

You're not going to get a very high degree of accuracy from a ~$100 test. They're not sequencing your entire genome for that amount of money. It is still good enough to determine the broad strokes of someone's ancestry, to within a few percentage points as you see in the linked article.

If you're willing to pay for full sequencing then you're going to get a lot more accuracy and repeatability. You're also going to pay a lot more for it than would be warranted just to satisfy your curiosity as to where your ancestors came from. These kinds of tests would really only be warranted if you have some rare, difficult-to-treat genetic disease. That might not even help you directly, but rather, be done to contribute to the body of scientific knowledge of the disease.


It is accurate and repeatable... to some point.

If you do it 10 times, you will always get the same buckets, but the percentages will differ back and forth for a bit. If you're goal it so find out your heritage, which these services are offering, then you are getting the right heritage repeatedly.

If your goal is to find out how many 0.1% of you is of an exact descendance, then you're going in with the wrong expectations.


Why? What makes you think your DNA is identical everywhere or that the test is perfectly accurate every time, or that even the concept of DNA-by-region is all that precise?


If you sign up twice, won't they match your account and merge the results specifically to avoid this problem?

I think part of why they used twins is to say its two different people so they'd have to treat it different.

I'm actually curious how repeatable the results are. If they are using sampling of portions of the DNA (are they?) how much variability is there in a single dna sampling enun through the algorithm 10 times? Similarly for how much variability in 10 dna samples from the same person?


> If you sign up twice, won't they match your account and merge the results specifically to avoid this problem?

Not if you sign-up with someone else's identity and pay with a different card. I very much doubt they go to any effort to verify the human the sample came from is the same human who paid and who the results are set to be sent to.


Or you could have a friend buy it then reimburse them. Or buy and swap a few times to see if it affects the results. Would be easier to test the theory if it wasn’t so expensive.


"You have another 100% match!"


`Twins find each other through DNA service 23andme` read the hackernews post title


>I thought it was pretty well known that these services are estimates only to a large degree? These services only take samples, they don't sequence the entire genome. Of course there will be errors when you have to extrapolate from a sample.

The credit given to the intelligence of society at large here falls somewhere between "cute" and "astounding".

I would recommend flipping through a tabloid and then considering such content sells millions of copies each print. And then reconsidering your thought that it's "pretty well known that these services are estimates only to a large degree".


Your post provides a great data point supporting your thesis, but the real problem with the intelligence of society is rational ignorance. People don't expend a great deal of resources trying to understand things that don't matter to them.

For example, you're happy to fire off about how dumb everyone is for reading tabloids because the finer points of how people unlike you consume media are not particularly important. And, likewise, the typical person buying a DNA test as largely a conversation starter isn't going to invest a great deal of effort to understand how DNA tests work.


Your post provides a great data point supporting your thesis, but the real problem with the intelligence of society is rational ignorance. People don't expend a great deal of resources trying to understand things that don't matter to them.

Tragically people are notoriously bad at long term predictions of what will matter to them and how that relates to a broad corpus of knowledge or habit of skepticism. The result is that people are so ignorant they’re under an illusion of being knowledgeable, and at that point they’re more or less a lost cause.

https://en.m.wikipedia.org/wiki/Dunning–Kruger_effect

It might help to know enough about DNA to wish to avoid these services, especially given how they cooperate with law enforcement. Mostly though, it’s just really hard to calibrate your corpus of useful knowledge from a position of profound ignorance. Plus a lot of people really are quite stupid, and ignoring that in favor of some superficially positive message about mutual respect is potentially dangerous.


Someone buying a tabloid doesn't necessitate that they believe what they're reading.

You could easily make the point that many people won't understand something without coming across as judgmental and elitist, but failed to do so.


I disagree. He didn't ask that "don't most people know you need to fill up your car with gas once the tank gets low?" He made it seem like the fact that there is a high degree of uncertainty/estimation in genetic testing is common knowledge. We're talking about a society where half the members do not believe in anthropogenic climate change (note that I do not state the _degree_ of the change) and sizeable numbers of people (millions) don't believe in vaccination.

He said "Of course there will be errors when you have to extrapolate from a sample." The common man doesn't know what extrapolate from a sample even means!

I'm sorry but suggesting society as a whole understands genetic testing methods and techniques is just so far absolutely removed from reality (you're perhaps justified if you work for Illumina or are a faculty member of a University) that the point needed to be made even if harshly. Maybe I just interact more with "non hacker news" types than the average poster here, but that post quite frankly seemed so absurdly out of touch as to remind me of the Lucille Bluth quote from Arrested Development - "I mean, it's one banana Michael. How much could it possibly cost? 10 dollars?"


I thought it was pretty well known that many tabloid readers read them the way one reads The Onion.

Sorry I couldn't help myself. :)


I did exactly this back when 23andMe switched to a newer chip. I signed up for a second account and matched with myself as a full brother. The old chip account had less data points (health items, etc) but most of the data was about the same.

My wife did the same with the same result, as did a friend who had purchased the original 23andMe kit and wanted to get the newer data from the new chip.

So while I don't doubt the accuracy of the linked article, my anecdotal experience is that the 23andMe DNA tests have repeatability at least.


On top of that even if they were able to sequence things perfectly the thing they are actually measuring is a very fuzzy thing. Like it's no surprise that one of the test had one twin from England but the other from Scotland/Ireland because people having been moving between those 3 countries for millennia so there isn't really a platonic idea of 'English DNA' it's just markers that tend to show up in England, but also show up near England but somewhat less and may also randomly popup other places, but usually not all of them in one other place.


Mine also changed after I complained that the results were obviously wrong. It indicated that several of my chromosome pairs came entirely from my father. Since that's generally impossible, it put the rest of the results in doubt.


how long have you been using an ad-blocker? perhaps the fine print says the expected accuracy is only guaranteed if they can source an adequate profile from data mining resellers?


It always surprised me to see them reporting results to a 10th of a percent precision (e.g. "2.3%"). I'd assumed that the best they could do on an individual was a couple of orders of magnitude less precise (e.g. 10%, 20%)


By day, I work with scientific equipment. Most people understand significant digits. But they also have conditioned expectations related to the number of digits that they see, and the quality of the equipment or measurement. People just like to see a digit after the decimal.


Yeah, "You're 76% German" makes my monkey brain think, "Come on, you're making that number up." But "76.2%" makes me think, "Ah, the miracles of science, time for a weizen of pilsner!"


> A single person can just sign up twice for these services and get different results

Do you have any personal experience with this? Or anything you can cite? I'd be interested.


Could I sign up 10 times and use statistical regression to get very accurate results?


Not necessarily. The results may only be as good as the training set that was used to produce the calibration (probably butchering the terminology). And the results that the twins got for their tests may have been "equally good" in terms of the reliability of the test.

The whole concept of geographic ancestry may have fundamental limitations at the level where people are quoting percentages.


Are there any services that sequence the entire genome/have a higher level of accuracy?


Dantelabs does. Note: I am not endorsing their service, I have no experience with them.


I wonder if there may be two issues here, one related to consistency and one related to twins. That is, you may not need twins to get differing results, but being twins may still be one of the reasons for differing results.

I'm not a biologist, but apparently it's not the case that twins have exactly identical DNA:

"Identical Twins' Genes Are Not Identical" ( https://www.scientificamerican.com/article/identical-twins-g... )


Can you explain the difference between sampling and sequencing the entire genome? I had assumed they did the latter but I guess the particular markers for diseases they are interested in are at known location that they can "jump to" somehow?


There are basically 3 approaches:

* Full DNA sequencing. This is a multi-million dollar effort taken on by large collaborations, usually to generate a reference genome sequence for a species or particular line. Typically a variety of methods are used, from 'shotgun' style illumina, 454 longer reads, and now some newer even-longer-read methods. Long reads are important for generating accurate assemblies. Generally, some tricky portions will need to be meticulously amplified and sequenced with traditional Sanger sequencing to clean up. This gives you a complete sequence, including all inter-gene sequences, repetitive portions, etc.

* RNA sequencing. This method is the core of many molecular biology labs; you take expressed sequence (i.e. RNA that has been transcribed from DNA), reverse transcribe it to DNA, and then sequence the DNA. This gives you sequence information about the expressed part of genes (this is limited, though, because during the assembly process you'll reject many reads that don't map onto the reference sequence in standard analysis) but mostly gives you an idea of the relative expression of different genes. When people talk about gene expression going up or down in certain mutants, conditions, etc., they're usually talking about RNAseq (or qPCR if only a few loci are involved).

* 'Sampling' in this case refers to chips with various DNA fragments on it. You pass the sample DNA over it, and compatible sequences hybridize together, which is then detected via florescence. These chips are designed to carry sequence fragments that vary in the population. Together, these variants form a DNA profile. The quality of this profile depends on the design of the chip. The sequences included are generally loci that are known to affect particular diseases or to be especially variable in the population for assessing ancestry.

In cost magnitude, this goes from millions to thousands to tens.


Your comment gives the impression that all DNA sequencing is a multi-million dollar effort. So, just to clarify: a modern, accurate (= 30x coverage) whole-genome DNA sequence costs less than 1000 USD. Adding long range information from single-molecule sequencing makes it only slightly more expensive.


I”m glad you added this and want to add further that GP’s impression is less than a decade out of date.

The price of a whole genome sequence has fallen dizzyingly fast, faster than any technology of which I’m aware. We’re close to the point where it’s priced like any basic blood test. Exciting times.


Think of a genome as 3 billion character sequence. "Sequencing" a genome gives you the actual content of the sequence. Think: "....GAAAAGATACCCACAGAGATTACAAAAC...." (I'll call this string FOO) Until recently, sequencing an entire genome was really, really expensive. I'm not sure what the price is now, but I know that it has dropped considerably.

When you sample a genome, you're checking for one specific string. Traditionally this is done by mixing a sample of the DNA with an enzyme that breaks apart a specific DNA sequence and then measuring whether the size of the DNA molecules have changed. Take the string that I mentioned above (FOO). Suppose that I mix it with an enzyme that cleaves along "GATTACA" such that "GATTACA" becomes "GAT| |TACA". If I mix a DNA sample containing the string FOO with the enzyme I just mentioned, then the molecule would be split into: "....GAAAAGATACCCACAGAGAT| |TACAAAAC...." If I mix a DNA sample that does _not_ contain FOO with the enzyme, no change occurs.

You know when you watch CSI shows* and the DNA evidence is presented as a bunch of light and dark bands? Those are produced by mixing the suspect's DNA sample with a standard cocktail of enzymes, treating them so that the DNA molecules are electrically charged (DNA might already have a charge, not sure), placing them at one end of a container of a standard gelatin with known pore size, and applying an electric field for a standard amount of time. The cocktail of enzymes cleaves the DNA at multiple different substrings and breaks down the DNA strands into multiple smaller substrings. Different size molecules will progress through the gelatin at different rates, so you can effectively compare two people's DNA to see whether the substrings match. You _don't_ know the person's genetic sequence, just whether or not his or her's DNA contains the same substrings as the DNA of the person that you're comparing it to.

* I'm assuming those TV shows present the DNA evidence this way. I don't watch them.

Disclaimer: I'm not a biochemist by any stretch. This is based on my memory from school, so don't take my explanation as absolute truth :-)


There are three techniques in play here: Sequencing, fingerprinting, and SNP sampling.

You have described sequencing and fingerprinting. These services use SNP sampling.

Sampling has been described several places in this thread so I won’t repeat that here;


I started typing an answer, but realized Wikipedia could probably do it better. In short, sequencing and checking for specific mutations are very, very different processes.

Sequencing: https://en.wikipedia.org/wiki/Whole_genome_sequencing

Microarrays: https://en.wikipedia.org/wiki/DNA_microarray


its like taking a low resolution photography of 23 chromosomes and X and Y plus the circular mitochondrial genome. The higher the resolution the more expensive it is and more data and more difficult to interpret. If the feature you interested in is large and unique then a blurry picture may be good enough for example recognize a human body from few yards away, as the features become more refined and more diverse like a face, you can still have little blurr but you need to recognize the facial dimensions extensively to be useful. And if you want to see skin lesions then you want a high resolution pictures to distinguish which type it is...

sampling is a high resolution photo with large holes in it, a full sequencing is a full photograph with varying resolution and quality etc. There are still some areas of the genome that are inaccessible even with full sequencing. Improvements are being made with long read sequencing.


I haven’t seen a sample report, but doesn’t this mean the outcome could be, “you may or may-not be predisposed to xxx disease”. We need to spend $99 to know that?


No, those are specific genes they test for.


Can the errors extend to incorrect biological parents?

I've heard stories of affairs being discovered through DNA testing. Curious if the denials might have validity.


It would take so many errors adding up that I think it would be very impossible. Has anyone ever had a real sibling not come out as around 50% related?


The situation would be a child finding out they share 1 parent with their sibling instead of 2. Curious if the errors can extend that far.


I'm not a biologist so I'm basically making this up, but my guess is no. I think only false negative errors are possible because of the way they do their sampling.

If two samples had enough similarity to say someone is a child of someone else, my guess is that's pretty accurate.


> Lack of oversight

> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.

> It's a situation Gravel finds troubling.

What is up with this incessant clamoring for a nanny to supervise every little thing?

This is published on a page where, thanks to "oversight", I have to click a little spam box warning that this page, like every other damn website on the internet, uses cookies. No thank you. I really don't want to pay more taxes for the Central Bureau of Making Sure You Know You Might Be 7% Less Eastern European than Your DNA Test Indicates.


That's overly reductive. There are people making lifestyle changes and healthcare decisions in reaction to the results of their DNA test, so the accuracy of these tests may be more important than "woops, we placed you in the wrong European ethnic group".

And I guess we remove all oversight now because of a bad example? Pack it in, FDA. We're done monitoring food safety thanks to cookie banners.


You're being equally reductive. Obviously all of this exists on a spectrum. Most reasonable people would agree that the FDA is a net positive. Most reasonable people would agree that the government monitoring every private conversation happening in the country and jailing everyone who tells a lie would be a net negative. The happy medium for government checks on behavior is somewhere in the middle.

Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside. People make lifestyle choices based on things they hear from palm readers - we don't use the government to regulate and test the validity of psychics. I think this falls into the same category.


> The happy medium for government checks on behavior is somewhere in the middle.

That was my point.

> Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside.

I think we read different comments; all I got was a snarky condemnation of government oversight.


> Most reasonable people would agree that the FDA is a net positive.

A 1974 study by University of Chicago economist Sam Peltzman concluded that since 1962 the new rules had reduced the rate of introduction of effective new drugs significantly—from an average of forty-three annually in the decade before the amendments to just sixteen annually in the ten years afterward.

... The severest criticism leveled at the drug lag is that without access to a drug available elsewhere, seriously ill patients will suffer or even die. Peltzman raised the subject in his 1974 study. He noted pharmacologist William Wardell's estimate that because the relatively safe hypnotic drug nitrazepam was not cleared for use in the United States until 1971, five years after it was available in Britain, more than 3,700 Americans may have died from less safe sedatives and hypnotics. After earning the Nobel Prize for chemistry in 1988, U.S. drug researcher George Hitchings of Burroughs Wellcome Company said of an antileukemia drug he helped develop before the 1962 amendments: "We went from synthesis to the commercial drug in three years. That is absolutely impossible today."

http://www.econlib.org/library/Enc1/DrugLag.html

At this rate, the benefits of FDA regulation relative to that in foreign countries could reasonably be put at some 5,000 casualties per decade, or 10,000 per decade for worst-case scenarios. In comparison, it has been argued above that the cost of FDA delay can be estimated at anywhere from 21,000 to 120,000 lives per decade. These figures would seem to support the conclusion that the costs of post—1962 regulation outweigh benefits by a wide margin, similar to Peltzman's results of a 4:1 cost-benefit ratio for the 1962 amendments.

"The Safety and Efficacy of New Drug Approval", https://pdfs.semanticscholar.org/feb9/53ab293f24a14b3b3d4b74...


Well let us not forget that not too long ago 23 and Me swayed a little too far into this area with warfarin sensitivity (circa 2014 I think).

https://www.fda.gov/ICECI/EnforcementActions/WarningLetters/...


This is the product they are offering, they're telling you which ethnic groups you match, this is what is being discussed, this is what is being criticized as inaccurate. What is being proposed here here is someone from the government to step in and look over their shoulder and make sure they approve of the methodology being used to generate the results.

If a company is making health claims there's already abundant regulation in place to prevent you from making unfounded statements. If you are making health decisions based on an Ancestry DNA test without consulting a doctor, you're clearly on your own.


they can't offer health results at all in most of europe and other countries. I don't think that's fair either, why is my government trying to protect me from learning anything about my genotype? Especially considering european countries that have socialized healthcare (and thus healthcare costs cannot be affected by letting this info out). This legislation does not make sense.

https://int.customercare.23andme.com/hc/en-us/articles/21769...


anyone who changes their lifestyle as the result of a faulty, voluntary DNA test deserves whatever consequences that happen


> What is up with this incessant clamoring for a nanny to supervise every little thing?

The incessant clamoring of these companies to sell themselves as a medical device platform.

Snark aside, it's fine that they have things set up the way they do for ancestry reporting. A couple of percentages here or there do not make a lot of difference when you are using it in a recreational use case. When you start getting into reporting potential medical associations with SNP variants which could affect clinical treatment decisions, it stats to matter a lot.


>> I really don't want to pay more taxes for the Central Bureau of Making Sure You Know You Might Be 7% Less Eastern European than Your DNA Test Indicates.

I'm not sure if you're serious, but 23andMe also offers tests for genetic risk factors of some serious diseases. I don't think people can verify the accuracy of these tests in their kitchen.


>What is up with this incessant clamoring for a nanny to supervise every little thing?

It is because corporations keep lying and behaving horribly.


What you call "nanny" others think of as civilization.

But if you want to experience complete freedom, you can always go out there in the boondocks, dance with the wolves. Let that experience teach you the value and importance of the "nanny".


Fill me in on what value you see on having taxpayer money go towards making sure novelty DNA tests are "regulated" (and what could you even meaningfully regulate?)


This is what's known as a compound question, and it's almost like a way of straw-manning someone. GP didn't say he wants to regulate "novelty" DNA tests. That's the spin you're putting on his position, and you've asked your question in a way that he can't answer it without tacitly approving of your spin. It's like the classic "When did you stop beating your wife?"

Are these tests just novelty tests? Are they intended to be? Is it clearly communicated to consumers that they are just novelty tests?

If they are just novelty tests, but they're not being upfront about that when they take consumers' money, is that acceptable public behavior? If that's not acceptable behavior, what is the right social response? That's the point in the discussion where GP appears to be.

If these are novelty tests, and are being marketed as such, would you mind linking to a page on, say, 23andMe's website that describes them as such?

If not - if, in your own words, these are "novelty" tests but the companies involved aren't being forthright about that - do you think a government response is so inappropriate as to dismiss it out of hand?


Well, you could mandate that they be called novelty tests.


Semi-related: I was thinking about getting one of the newer iWatches, the one with the heart-rate monitoring and all that jazz. To me, those features are really cool and future-y. So, I waltz on down to the store and try one on. It reads my heart-rate and it seems to be fairly accurate. Ok, well, let's test it a bit. I start running in place. The heart rate goes up. I then just stand there, after a bit, the heart-rate goes down. Nice, the thing actually works! Well, they had a few of the iWatches sitting there, without anyone else taking a poke at them. So, I strap all the free ones on the arm.

Well, they all gave different readings. I repeated the running in place test. Some went up and down much more than the others; maybe 10 bpm more. I tried re-arranging them on my arm and doing the running in place test again, you know, maybe it only works on the wrist since the blood is closer to the surface there? Nope, the readings were all over the place still. It was not super scientific of me. I mean, I was some strange person running in place in a store with a bunch of iWatches strapped to my arm.

Maybe the watches have some sort of learning algo in them that was getting messed up with all the folks trying them on. Maybe I was wearing them 'wrong'. Maybe I should have waited 10 minutes between jogging sessions. I don't know.

But, to me, these things have a long way to go. A lot of the bio-tech and bioengineering out there seems to be a bit of snake-oil right now. I really do want it all to work, I think that would be a great boon to us all [0]. If something like the iWatch can't give repeatable readings in nearly any way, then it's all just a gimmick and not useful.

[0] I mean, can you imagine if Theranos' tech actually did work?! That would change medicine for the better in incalculable ways. I can see why that company's plan was so intoxicating.


When I tried on an Apple Watch at an Apple Store, I noticed that the heart rate it displayed was definitely different from mine.

I mentioned it to the employee who was manning that table, and she told me the demo watches had some pre-programmed data for demo purposes.

I was puzzled by this since heart rate monitoring is a key feature users care about. It occurred to me that heart rate data of everyone who has tried on the watch might count as the kind of medical information you shouldn’t publish on demo watches.

Imagine ridiculous stories like “CEO of X’s heart rate was 130bpm when he tried on the new Apple Watch, says journalist who tried it on right after.”


> It occurred to me that heart rate data of everyone who has tried on the watch might count as the kind of medical information you shouldn’t publish on demo watches.

That's an overly generous explanation. Fitbit-like devices are notoriously inaccurate[1], and the simplest explanation seems to be that their inaccuracy is unintentional.

[1] https://www.independent.co.uk/life-style/health-and-families...


The Apple Watch is in a different regulatory space than the devices listed in that source, since it's cleared as a class II medical device by the FDA. HIPAA laws will apply in some use situations, so their lawyers will have thought about it and made recommendations. (Lawyers aside, the feelings of both the medical community and Apple are generally that you shouldn't leak customer information. This seems like the kind of data that people would accidentally leave behind at the store without thinking about it, which doesn't feel right.)

https://hipaahealthlaw.foxrothschild.com/tags/class-ii-medic...


so we need to find someone with access to 2 identical watches, perhaps someone who bought some for the whole family? instead of strapping all to the same arm, just strap one on each wrist (assume the "left" heart's rate is approximately equal to "the heart on the right").

Are we claiming that every product needs 2 FDA applications: one for the end user and one for a special run of demo watches? I believe (but did not check) that demo watches would be either exempt or go unnoticed considering the amount of data it could gather...


Heart rate data gathered from a store demo device without any personally identifiable information isn't subject to HIPAA rules. If it was reading inaccurately then most likely the band was loose or the sensor was dirty.


Well, I'll be damned! Thanks for the explanation, I really appreciate it. I'm still wanting to buy one, but now I don't know what to think. I wonder how I am supposed to try one out before I buy one and to make sure it works.


In a study two years ago Apple Watch was the most accurate tested device with a median error of 2%.

https://www.macrumors.com/2017/05/24/apple-watch-heart-rate-...


I guess you would have to bring a proper heart rate monitor and compare the two readings.

Although people might give you strange looks :)


You can buy any Apple device, use if for two weeks and return if for a full refund if you don’t like it.


Wrist worn optical heart rate monitors are actually very accurate, under normal circumstances when worn properly. It's easy to verify accuracy yourself by just counting your own pulse rate with your finger. Try it.

Wrist devices will give inaccurate pulse counts if the band is too loose, skin is dirty/wet, or your arm is moving around quickly. If you need accurate numbers in all circumstances then a chest strap which detects electrical signals is a much better option. I usually wear a Garmin HRM-Tri strap while exercising and it transmits a wireless signal to my fitness tracker.


Manually (as in manus, the hand) measuring your pulse while not at rest (sitting down, in a calm environment) is so error prone, it's almost useless.

The only proper comparison would be with a chest strep, which are much more accurate.

Also, it really differs per person and how you wear the watch. Just wearing it a bit differently can change the results significantly.

Sweat also gets them messed up easily.

All reasons why, if you want accurate data for workouts, you should always use a chest strep. Even if they are much more awkward.


Anecdotal, but I got an Apple Watch because I never liked the squeezing feeling of the chest strap of my Suunto. I put them to test for the first couple weeks and both at rest and while running the difference in the HR reported by both was less than 1bps.


So I ride bikes and always monitor my heart rate while working out.

I've always used a chest-based heart rate sensor until I started trying wrist-based about 4-5 years ago. I've used countless brands and models (ranging in price from about $49.99 up to about $399) on my wrist - and literally none are as accurate as the chest-based strap. The chest strap responds immediately to increases and decreases in heart rate. Whereas the wrist sensors always seem to lag, sometimes by up to a minute or so.

The wrist-based sensors always seem to differ depending on where on the wrist they are worn - high, low, inside of the wrist, outside of the wrist, etc - by maybe 5 bpm or so.

In either event, if you really want to test the accuracy of the wrist-based sensor, you should really compare it to the output of a chest-based strap.

In fact, the results will largely be personal - because the electric signals may differ per person greatly in different areas of the arm.

Needless to say, my apple watch is worn daily, but when riding, I have gone back to a chest-based strap.

That all being said, there is a guy that reviews these gadgets and will often wear 2-4 sensors while working out, and post an overlay of the results from his workouts. The baseline should generally always assume that the chest-based straps are accurate. His site is called DCRainmaker, if anyone wanted to look up his comparisons/reviews.


I wonder if the difference in results is partially due to a difference in measurement strategy: a naive way is to wait one minute while counting the pulses which will necessarily be slower in response (it could still update the screen at a faster rate of once per second by having 60 different integers but each integer gives the count displayed on the screen when it's last second has passed, but that would still have a slow response in the sense that given a hypothetical heart rate change from a fixed constant initial rate to a new fixed higher rate, like when you got scared by say unexpected fireworks, it would still take a minute for the final rate to show up).

A faster response could be got by measuring the time between heartbeats (probably a low but even number of beats).


The wrist lag is the biggest problem I have with my Fitbit wrist band. It is OK for looking backwards of how my overall work out went, but it is useless for trying to hit a target heart rate.


So, I strap all the free ones on the arm

I can't say that I know exactly how these things work, but I wouldn't be surprised if they were built to work with the blood vessels and skin thickness of a wrist, where watches are worn, rather than the skin thickness+hair+different vessel locations of a forearm.

Different parts of the body are used for different things. It's why blood oxygen monitors that work on fingers don't work on toes.


Yeah, I tried to move them about over my arm, thinking exactly the same thing as you just did. It gave me all kinds of results. Another commentor points out that the data is 'demo only' data on the in-store watches. The reason for demo only data is that the data might be considered 'medical' information and, I assume, therefore covered under HIPAA. Exposing that to just about anyone that can come in the store and see the record of the bpm is, rightly, very illegal and unethical.


The research I've read [1] says that heart rate monitor in general and Apple Watch in particular should be pretty accurate. If you have darker skin or wider wrists that accuracy may be diminished.

[1] https://www.mdpi.com/2075-4426/7/2/3


Thanks for the link!


Disclaimed: I used to work in this field, but not on the Apple Watch.

The wearing a bunch of them at the same time is going to be wrong, yeah. Wrist mounted fitness devices are calibrated to read on the top side of the wrist. Even reading on the bottom side of the wrist would be a bit different, way up the arm? Very different.

It is possible to make accurate devices that read there, they just need to be constructed to do such!

Wearables are also meant to give an accurate rating over the long term. If you go running for 30 minutes, are the overall calorie burn, average HR, max HR, and min HR, all correct? Tracking in perfect sync with the heart is nice, and of course everyone tries to do that, but if the number is off for a second here or there, well so long as the overall result is the same the usefulness hasn't really changed.

Finally, due to the way optical HR works, random unexpected, movements are really hard to deal with. It can take some time for an HR sensor to lock on to the heart rate. This isn't too long (measured in seconds, and each generation of devices is getting better and better), but it relies on regular patterned movement, such as arms pumping while running or walking, the regular motion of a bench press, etc.

Stopping and checking the HR value every few seconds to see if it is correct is literally the worst case scenario for a lot of wrist mounted sensors!

Ground truth is strapping a heart rate strap to your chest, putting on the wearable, going for a run as you normally would, and comparing the two at the end.

Multiple sources have done this, and found the Apple Watch to be best in class when compared to directly measuring HR.

tl;dr: Stopping and checking wrist mounted devices can confuse them, Apple Watch, and everyone else in this field who is serious, are tested against HR straps and they come within spitting distance of accuracy.


This is a very interesting story and I imagine there is some amount of ML used to gradually improve the accuracy of results for a specific user.

Unrelated: It is called the Apple Watch. :)


It's not an interesting story, and his method of testing was severely flawed.

There are tons of people that test these devices against chest straps online, and almost all of these bands give results that are surprisingly accurate.. or at very least about what you'd expect out of a wrist based monitor.


There's no ML. It's a direct measurement.

ML only works for these types of measurements when you have some kind of reliable source of truth or fitness function to compare against.


I have the first apple watch. (I bought the newest but haven't used it yet still boxed).

What I found was that the heart rate correlated well with taking it manually and/or a finger heart monitor (pulse ox cheap from Amazon).

But with both devices you could actually tell when it was clearly wrong independent of comparison. The point is you more or less (at least I found) could figure out when it is right and wrong just based on past use and what you would expect (feeling your pulse). So if it was off it was grossly off and obvious. That is my experience with the watch that I have had for years. Either 'correct' or 'vastly off and easy to tell that is the case'.


Some of those things stop working correctly when your skin gets damp. Think about that for a second.

An iWatch is not the pinnacle of wrist biometrics, however. I'd be a lot more concerned if a high end Garmin got it wrong, that's their whole business.


Part of how you're supposed to use the dedicated higher-end models (I have a Garmin Forerunner 645) is that it's supposed to be pretty tight against your wrist. Even if you "wear your watch tight" it's probably not as tight as you're supposed to wear it for fitness purposes. If at any point you can't feel it or you forget you're wearing it, it's definitely not tight enough. This helps limit optical issues with movement but also largely eliminates the impact of sweat on the reading.

Still not nearly as accurate as the chest-based models as mentioned elsewhere, but for 95% of people it's plenty.


Garmin and Apple wearable devices use basically the same optical heart rate sensors. They don't make their own. Every company in the industry buys from the same couple of suppliers which have developed the technology.


That's surprising, since the heart rate monitors on many phones are quite accurate. Many will actually give an ECG-like trace indicating brief changes in blood pressure. I don't see any good reason they shouldn't be extremely accurate.


Well unless you're wearing a cuff you're not getting any blood pressure readings on your phone, and any "ECG-like trace" is a doodle at best as there's no electrical monitoring. Even chest straps that measure electrical output don't give you an ECG display.


I'm not suggesting that absolute blood pressure is being measured by a phone PPG, only relative changes.

Far from being just a doodle, the PPG readout ("ECG-like trace") comes from changes in skin reflectivity that indicate relative changes in blood pressure. [1]

Obviously this isn't as good as an actual ECG (and can be disrupted by movement), but it's far more information than is needed to accurately track heart rate. These devices can be as accurate as a pulse oximeter [2], though they don't account for the movement of the user.

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4822420/ [2] https://www.ncbi.nlm.nih.gov/pubmed/28464700


Disclaimer: I currently work for a genetics genealogy company mentioned in this article. My comments are my own. Some relevant previous comment threads of mine about other DNA related articles: [0], [1], [2], [3]

Personally, I would love to help create legislation aimed to protect consumers from perceived and actual problems related to DNA sequencing and analysis. I don't know how though and I fear it'd fall to the back burner in the current US political climate. Our customers are not only US but international though and that presents yet another dimension to the challenges therein.

This article states:

> Whatever your ancestry results, don't get too attached to them. They could change.

That is absolutely correct. I've stated why in a previous comment [1]; Some DNA analysis software employ stochastic algorithms. That means that the answer they provide can be different if run more than once.

The article also states:

> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.

That is also correct. I've heard peers state that the industry isn't regulated outside of the field of medicine during product discussions. I think that's something which needs to be addressed: leaving it unaddressed can encourage predatory business behavior.

[0] https://news.ycombinator.com/item?id=18196717 [1] https://news.ycombinator.com/item?id=18196984 [2] https://news.ycombinator.com/item?id=18564380 [3] https://news.ycombinator.com/item?id=18569659


> there is absolutely no government or professional oversight

They are not allowed to sell health-related reports anymore in all of europe and other countries. While i think that some regulation and standards would be good, banning it overall was a dumb measure.


Can you quote a specific law for that?



Excellent, thank you!


Well 23andme was specifically reprimanded by the FDA for their health related information and blocked from publishing future health advice, until they got limited approval after finally complying with regulation. So it’s inaccurate to say that there is no oversight, although I agree that the current oversight is clearly insufficient, in addition to being post-hoc.


I wouldn't mind paying a fair sum for a service that would:

(1) reasonably guarantee that my data not only won't be distributed - but will be destroyed (on their side) within a short time frame

(2) give me access to all of the raw data and methodologies used in their analysis

Does anyone know of such a service?


The only reasonable way of achieving those goals would be to allow totally anonymous submissions. Mail in a sample and simply have a code that you keep to check on results once they are processed.

The fact that none of these services allow for anonymous submission should be a huge tell about their real motivations.


That would help, to a degree.

But ultimately any kind of DNA analysis (that exposes the raw data) is intrinsically de-anonymizable, no?


Anonymous submissions are in theory somewhat feasible, albeit cumbersome. You can by the 23andme kits with the full fee paid from certain pharmacies (I think Target had them) using cash. This means you don't have to put down a credit card when signing up. I leave the rest to the reader to work out wrt anonymizing themselves while signing up to the website.


Err, isn't the first thing they do is link up the new submission with every other submission ever received in the past or future, almost none of which are anonymous?

Sure, you can be anonymous, but you're the son of John and Jane Tompkins, the brother of Emily Tompkins, the cousin of Mary Smith, and the father of Dave and Buster Tompkins.... huh, I wonder who you are.


Very good point. Added requirement: Make sure all your ancestors and extended family are dead and did not do 23andme before their passing.


Unless things have changed, this is exactly how 23AndMe works. You get a kit and send it in, then you just have an ID number that ties that kit back to you for setting up your account.


That might work until you have relatives who did not do it anonymously...then you get outed by the DNA istelf. A service that destroyed your sample and record after processing would help in this regard.


23andme does destroy your samples, and the genotype tests are not very useful for determining paternity.


That would also help against the (hypothetical?) possibility that data mining sector creates "DNA testing" front companies, which do no testing whatsoever, and simply sell you a pretty printed conclusion from your cyber profile... without oversight anything goes


That would fall under "unfair and deceptive business practices" which there are laws against.


obviously, but a law alone can only protect you when the deception is detected. the proposal to somehow fully anonymously (so somehow also not through the usual mail system) submit the material and code would be a preventive measure


> none of these services allow for anonymous submission

You're incorrect. At the company I work for, which is one of them mentioned in the article, a customer pays for DNA sequencing and analysis. Who pays for it is often separate from who's being analysed. We often have customers who pay for products for their family, friends, or otherwise. The name on the DNA sample is not verified in any way shape or form (and how could we verify it beyond just billing information?) You can opt-in or opt-out of public matching. You can download your raw data anyway (GDPR requires this if you're an EU citizen).

> should be a huge tell about their real motivations

I would like it if you'd elaborate on this.


> I would like it if you'd elaborate on this.

I don't have an opinion about this, but they're probably implying that the business model is the same as Google, Facebook, et. al., which is selling user data to third parties.


1) Pay for sequencing, get raw FASTQ or CSV files. 2) Run your own analysis. Open source industry standard software exists.


Is this actually an option with the popular companies?


It's not advertised as such but yes, several companies allow you to obtain the "raw data" from the sequencer. The raw data is what gets fed into the analysis software which produces relationship and heritage estimates.


It's also worth noting that if the company sells analysis products to EU residents then GDPR may require those companies to make the raw data available to the customer anyway.


Might look into Dante Labs. I was fairly happy with their privacy policies and ordered a kit during black Friday for 200USD


Were you able to get the results from Dante yet? I am curious to see how they differ from 23andMe in report quality.


Don't most services provide (2)? You can download the raw data. It's not your full DNA sequenced or anything, they only do genotyping, i.e. only test for genes that are known to differ between individuals, I don't think there's a consumer service that does full gene sequencing for a reasonable price.


same here. I really want to learn more about my past and also learn about my DNA (and all there is to learn with that data), but the invasion of privacy by such companies is just unacceptable (hence my refusal to take such tests).


Misleading headline in my opinion...

Carly's results have a larger 'Broadly European' percentage, but that is a catchall that includes: Italian, Eastern European, Balkan, French & German, Iberian (Others category), and Broadly Southern European (Others category). Therefore Carly's 'Broadly European' traits could be used to match Charlsie's results.

It is unfair to call the 23andMe results 'different'. They are getting the same results, however one twin is getting more specificity.


Why do people assume ancestry results from these DNA tests are "accurate"? Human ethnicity is a continuum, not distinct pools of genes. We are applying labels conceived by modern nationalism to several thousands of years of migration and intermarriage. It would be great if people stopped treating ancestry results from these tests as authoritative reference.

There was a controversy about Korean & Japanese ancestry results from 23andMe where ethnic Koreans are sometimes classified as >30% Japanese. These two groups shared common ancestors since ancient humans migrated via the Korean Peninsula to the Japanese archipelago centuries before the modern nations of Korea and Japan came into existence. Slapping either one of these specific labels on shared ancestry is wildly inaccurate and short sighted.


Clumps appear in a principal component analysis, people just want to know how they look on one of those.


> Why do people assume ancestry results from these DNA tests are "accurate"?

Years of highly visible advertising by these companies which implies they're accurate? All of those “discovery your ancestry” posters they put up around here had specific countries and locations.


Agreed. Just a quick look at the 23andme site doesn't show me anything that says they are "estimates" or "not accurate".

The landing page shows me this headline "This new year, commit to a healthier you - inspired by your DNA." I'm sure they have it legally covered somewhere in their TOS but it's still not good if these are inaccurate or estimates.


Which could potentially even just be explained by something as "simple" as cohort testing to show more specific breakdown vs a more regional view


Their results seem well within the margin of error I'd expect out of a $100 DNA sequencing service. It's incredible that this kind of thing is now available at this price point.


The great thing is that their dnas were shockingly similar. It means the sequncing is valid. Now the ancestry composition thing is known to be a nonstandard procedure , just look at their research publications.


I'm imagining some kind of machine-learning training set based algorithm for determining the ancestry percentages. Even at 99.6% difference (due to mutations over time or sampling error) over 700k samples there's gonna be ~300 sequencing differences between twins. Enough to make their ML system draw different inferences for sure.


23andme has some publication about their method. the thing is , they have a lot more data than what is available in public repositories so they basically established their own system.


Yes, this is an upside down story.

If the post analysis shows nearly identical DNA that's great.

The whole '% from this or that country' has to be necessarily fuzzy, because that's just a probability game anyhow.


it is. and tbh ancestry is the least interesting part for me, since i more or less know my ancestry . however, the very distant cousins that i have found in US and CA are very excited to know where they came from. This must be a mostly american thing.


> This must be a mostly american thing

Part of the old-school American tradition many regions (mostly rural, which is where over 80% of residents lived at one time) was to have a bunch of kids and encourage them to spread out as far & wide as they wanted; this still continues in some places. The logistics of 'keeping in touch with family' have some pretty hard limits past a certain point, and recordkeeping can be haphazard/fragile.

A great many people in North America have significant trouble tracing their ancestry back past their great-grandparents.


The raw data is what's being called "shockingly similar" by the expert quoted in the article. He considers the final result confusing.


the raw data is what matters. the rest can be corrected


Not surprised. Some consumer DNA testing companies can't even tell when the DNA submitted isn't human: https://www.nbcchicago.com/investigations/home-dna-kits-4812...


Haha, very interesting. I've always wondered what would happen if I sent in a sample from my dog.


99.6% is still awfully dissimilar, depending on what that number means. Thinking about single polymorphisms, of ca. 3 billion bases, that is still 120,000,000 SNPs between twins. Of course, they aren't doing full genome sequencing, and they are probably targeting loci that are know to be more divergent. Still... that similarity seems awfully low for twins. I wonder what they typically find for similarity between any two individuals. I also wonder if their lab is a little disorganized....


Current estimates are that there’s a roughly 1 in 1000 difference between two individuals purely due to random mutations in the germline (or, put differently, the DNA replication error rate in human cells is about 1 in 1000). This would imply 99.9% expected similarity between two identical twins. However, this is in the germline. Somatic cells accumulate mutations over their life time (since they, too, divide and therefore need to replicate their DNA, and are furthermore exposed to environmental stress). In addition, the tests work off saliva samples which are mixed with all kinds of stuff in the mouth, and degrade slightly during transport. Lastly, there’s an inherent replication variability in the genotyping process (though this is very small).

This back of the envelope estimate suggests that 99.6% similarity for somatic samples of identical twins is not unexpected.


This is not accurate. Coarsely, there are roughly 3 million SNPs (out of 3 billion total haploid bases) per individual. When we talk about individuals "sharing" DNA we mean these variants and not the 99.X% of DNA that we all have in common. When we refer to random mutations in the germline (i.e. de novo variants) current estimates of de novo variants per individual are in the low hundreds.

The other factors you discuss are more likely to cause issues. One major one is the error rate of the microarray because of the high number of genotypes being assayed. If the chip has an error rate of 0.1% and 500k tests are performed you would expect 500 mis-calls. I used to work for Illumina, who makes these chips, but I worked on the DNA sequencing side of things so I don't know the actual error rate off the top of my head.


Right, de novo mutation rate is much lower (even lower than what you wrote, just slightly above 1e-8 according to deCODE). The 1/1000 rate is the average difference between two random individuals (not twins), which corresponds to your 3M SNPs per 3B bases.


Yes, but you referred to them as random mutations, which is not the case. They are inherited variants. The DNA replication error rate is nowhere near 1 in 1000.


Hence my correction. Without looking it up now, the actual replication error rate is probably close to the deCODE rate for de novo mutations (modulo gamete selection, so probably higher than that).


Something tells me that the 1/1000 error rate you cite isn't truly accurate, or at least doesn't apply to all portions of DNA equally, because in almost every individual that would result in many SNPs with bad consequences since we have so many subsequences coding essential proteins


Article said identical twins. So I'd expect around a couple thousand SNP differences. Each cell devision has about 30 mutations. Embroyo bifurcated within a dozen or so cell divisions. Dr. Hayflict found human soma cells live aboutb50 divisions.


Yes, especially since we estimate humans and apes share near 99% of their DNA sequence.


They don’t, this number is extremely misleading. Roughly 97% (I think is the current estimate) of their protein-coding DNA is equivalent, but not 1:1 identical.


Not the same kind of 99%.. For (fraternal) siblings you'll get around 50% or thereabouts.


No, that’s also incorrect. The real number is way higher. They share 50% of alleles, and hence 50% of variants. But a lot of the DNA within two distinct alleles will be identical, except for (say) a single nucleotide difference.


Yes, that's actually what I meant to say.. couldn't remember the right terminology at the time. The point stands though, I think.. the number "99.6%" for identical twins isn't measuring the same as "98.5%" (or so) for humans vs chimps.


Yeah Im a bit surprised about that, even given de novo mutations.


It represents the underlying error rate of the assay.


Which seems pretty reasonable for a direct-to-consumer, cheap test, right? Noisy sensors = variability in reports.


It's totally reasonable (and in line with my knowledge of a similar technology, DNA sequencing). Often the stochasticity comes from biomolecular processes rather than from the optics of the instrument.


Translating into IT / tech terms, people are VERY used to the concept of two .wav files sampled from the same CDrom being bit identical and useful in a court of law to prove both .wav files came from exactly the same music cd.

However people are really worked up about a new online service that encodes multiple analog music sources into lossy mp3 samples using different analog encoders each time, and the service sells an "identify matches similar to this song" as a service that people think is the same level of exactness as the old fashioned court-of-law .wav file bit by bit comparisons, but its actually more of a lossy best guess pattern matching service instead. Yes they DO use the same general technology, mostly, but its quite different in purpose and outcome.

An even better analogy is we are VERY used to an online service that OCRs a scan of a music CD and outputs the musician name every time based on OCR of the disc itself, and now people familiar with that technology are VERY confused by an app that listens to your cell phone mic and often squirts out the musician name based on that raw analog sound sample.

Its more a miracle the new tech works at all, than a scandal that both business models aren't identical in accuracy.


I wonder why they didn't mention male microchimerism, a still poorly understand phenomenon, in which male DNA is found in females. Possible sources of gene transfer are " unrecognized spontaneous abortion, vanished male twin, an older brother transferred by the maternal circulation, or sexual intercourse"

https://www.ncbi.nlm.nih.gov/pubmed/16084184/

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3458919/


I'd like to get this done for myself and my parents (to distinguish where the different traits actually come from), but I'm a little wary of putting this sort of information out there when we don't know how it will be used in the future. And by whom.

Are there services that you can pay extra that will do a full DNA sequence and have access to 23andMe's comparison data?

Or am I being paranoid?


From the article (and the company themselves):

"The company said it approaches the development of its tools and reports with scientific rigour, but admits its results are "statistical estimates."

It is very likely that there's a fine print on their procedure that everything is a Guesstimate and while their algorithms will process things the same for every sample, there's just too much variations that are unaccounted for even in such controlled environments.

What would surprise me for example is if someone sent in their DNA for analysis twice on the same company and get varying results. That would mean that the process itself is not well-established.


DNA in Europe is all mixed up. A lot of this is due to the Romans. There were also a lot of other migrations and displacements.

I was watching a BBC documentary about how Rome influenced the history of Scotland. The announcer, who has a very pronounced Scottish accent, took a DNA ancestry test. The results (IIRC) indicated Germany, Italy, and eastern europe. (The announcer interpreted this last one as Dacia) Come to think of it, he doesn't look so different from Metatron.

The twin results not matching is kinda disturbing, though.


I can't get past the sub headline: > Twins' DNA 'shockingly similar'

Identical twins are supposed to have identical DNA, right? Or did I really misunderstand a lot of school.


No amount of schooling could have truly prepared you for precisely how similar twins' DNA would be. Shockingly similar.


As I understood it he expected the different results were due to contamination or other errors resulting in compromizing the raw data to some degree. The shocking part was that the raw data was ok while the results weren't ok to the same degree.


The twins DNA is "nearly exactly" the same is an oxymoron. Nearly is not an exact. The twins have "virtually identical" ancestry profiles yet, their profiles are "shockingly similar"

They are comparing a low resolution map the genome. The genome is 3+ billion nucleotides, only 700,000 (0.0002%) is being compared to find 0.04% difference! that is shockingly similar!


Techniques used to read DNA only read short segments, from which larger segments are inferred.

Not surprisingly, the results are not reliable.

Feed the unreliable result into any classification algorithm, and you still get unreliably results (actually I was expecting worse)

Not only that, but you only read small parts- SNPs. So there is no bijection; just a rough correspondence. On the bright side, it's good for privacy.


I think you underestimate the quality and power of short read sequencing. Most well-designed sequence-based genotyping assays take into account the length of the read and are easy to read out. (You are thinking of issues with copy number variation and perhaps assembly of longer genes, which do have issues). The sticky part is the analysis--how realiable are your databases and your algorithm for representing the mix of SNPs and INDELs as %European...that is the question.


> Techniques used to read DNA only read short segments, from which larger segments are inferred.

I think you’re describing short-read sequencing here but this isn’t what any of these services does. They do microarray genotyping rather than sequencing. And in neither case is there “inference”. True there’s something called imputation but it works quite differently and I don’t think (though I don’t know) that 23andme uses it for ancestry. I’m also not sure what you mean by “bijection” in this context.


They really should have done the obvious quality control test: each of the twins should have done each DNA test several times, at least twice. Most comments seem to assume that the test samples were uncontaminated and that the sequencing was correct. Redundancy will help identify the source of the differences.


I don't know if any company provides that, but I would like to have access to my raw data (I guess, several Gbs worth or DNA pairs information) and use a different 3rd party of my choice for the analysis. (This way there would be a lot more competition for various ways to look at data).


TFA seems to be making a mountain out of a molehill to be sensationalist.

I don't think anybody's ever claimed this type of mail in test is 100% accurate, and most of the results were within a couple percentage points of each other.


What are the chances that these two people are not identical twins? i.e. fraternal. That could explain the differences in results from the same company. But they should also be quite trivially declared siblings in that case.


Mistaken zygosity is quite rare, in the single percent range, after doing some tests like blood groups or hair whorls. In this case, the DNA overlap is vastly higher than would be expected for siblings:

> According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.

If they were siblings, ~50% of the (several hundred thousand) SNP calls would be the same, not 99.6%. (Which is about what you would expect from identical twins, since that's where either one disagrees with the other, and errors usually don't strike twice, so a 99.6% overlap roughly implies a per-SNP accuracy rate of sqrt(0.996) = 99.8%, which sounds reasonable given the difficulty of sequencing and the inherent noise & randomness.)

On a fun side note, identical twins are not exactly 100% genetically identical, which is how rapist/murderer identical twins are being convicted these days based on DNA evidence; but there are so few unique mutations to each twin, and the new mutations wouldn't be on the common-SNP tests, that you have to do the equivalent of like 4 whole-genome sequences (IIRC, you need coverage of ~120x vs the more usual WGS of 30x) to get enough evidence to override the prior of any difference being just a repeated sequencing error, and it's very expensive.


> According to the raw data from 23andMe, 99.6 per cent of those parts were the same

99.6% is hardly an impressive figure. I would expect 99.9999% between twins at least. Chimpanzees and humans already share about 96% of genes.

https://news.nationalgeographic.com/news/2005/08/chimps-huma...


It makes sense, what I was wondering though was about the difference if any of the raw DNA (SNP) data they provide.


I would expect this level of variance in tests given the original twin genomes are not identical.


Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: