I Had My DNA Picture Taken, With Varying Results
Ozier Muhammad/The New York Times
Kira Peikoff, 28, had her DNA tested by three direct-to-consumer companies, and the results didn't agree.
By KIRA PEIKOFF
Published: December 30, 2013 421 Comments
I like to plan ahead; that much I knew about myself before I plunged into exploring my genetic code. I’m a healthy 28-year-old woman, but some nasty diseases run in my family: coronary heart disease,rheumatoid arthritis, Alzheimer’s and breast cancer.
Pathway found that Kira Peikoff had an average genetic risk of psoriasis, top, while 23andMe assessed it as higher than average, and Genetic Testing Laboratories as low.
Readers’ Comments
Readers shared their thoughts on this article.
So I decided to read the tea leaves of my DNA. I reasoned that it was worth learning painful information if it might help me avert future illness.
Like others, I turned to genetic testing, but I wondered if I could trust the nascent field to give me reliable results. In recent years, a handful of studies have found substantial variations in the risks for common diseases predicted by direct-to-consumer companies.
I set out to test the tests: Could three of them agree on me?
The answers were eye-opening — and I received them just as one of the companies, 23andMe, received a stern warning from the Food and Drug Administration over concerns about the accuracy of its product. At a time when the future of such companies hangs in the balance, their ability to deliver standardized results remains dubious, with far-reaching implications for consumers.
My experiment ran into hurdles from the start. After I ordered 23andMe’s saliva test kit, which for $99 promised a report on more than 240 health conditions and traits, it turned out that I could not legally send it in; the New York State Health Department forbids any labs that lack a state permit to accept specimens from a health-related test. Luckily, my in-laws mailed it from their home in New Jersey.
Then I learned that the other two companies I planned to approach were no longer offering genetic testing. Additional research led me to two more: Genetic Testing Laboratories and Pathway Genomics. G.T.L. charged $285 for a report on 25 disease risks, and required a professional sample collector to draw blood; Pathway charged $399 for a report on 24 disease risks. (In 2010, Pathway planned to sell its saliva test kit at Walgreens, but abandoned the idea after the F.D.A. challenged the sales. Now Pathway requires a doctor to order a kit on a patient’s behalf.)
After my tests had been sent, I braced myself for the revelations about my DNA. It took about two months to receive all the results, and when I did, the discrepancies were striking.
23andMe said my most elevated risks — about double the average for women of European ethnicity — were for psoriasis and rheumatoid arthritis, with my lifetime odds of getting the diseases at 20.2 percent and 8.2 percent. But according to Genetic Testing Laboratories, my lowest risks were for — you guessed it — psoriasis (2 percent) and rheumatoid arthritis (2.6 percent).
For coronary heart disease, 23andMe and G.T.L. agreed that I had a close-to-average risk, at 26 to 29 percent, but Pathway listed my odds as “above average.”
In the case of Type 2 diabetes, inconsistencies on a semantic level masked similarities in the numbers. G.T.L. said my risk was “medium” at 10.3 percent, but 23andMe said my risk was “decreased” at 15.7 percent. In fact, both companies had calculated my odds to be roughly three-quarters of the average, but they used slightly different averages — and very different words — to interpret the numbers. In isolation, the first would have left me worried; the second, relieved.
Medical ethicists and other experts have a different kind of worry about results like these: a lack of industry standards for weighing risk factors and defining terminology.
“The ‘risk is in the eye of the beholder’ standard is not going to work,” said Arthur L. Caplan, director of medical ethics at the New York University Langone Medical Center. “We need to get some kind of agreement on what is high risk, medium risk and low risk.”
Several other problems may account for my discrepancies. The genetic testing that these three companies offer is premised on reading segments of DNA called SNPs (pronounced snips), for single nucleotide polymorphisms. But these segments, which have been linked to diseases in research studies, vary among people.
Scientists have identified about 10 million SNPs within our three billion nucleotides. But an entire genome sequencing — looking at all three billion nucleotides — would cost around $3,000; the tests I took examined fewer than a million SNPs.
“Imagine if you took a book and you only looked at the first letter of every other page,” saidDr. Robert Klitzman, a bioethicist and professor of clinical psychiatry at Columbia. (I am a graduate student there in his Master of Bioethics program.) “You’re missing 99.9 percent of the letters that make the genome. The information is going to be limited.”
Companies choose which SNPs to read. By comparing the technical reports provided with my results, I found that my tests sometimes relied on different SNPs to assess the same condition, like coronary heart disease. Each test studied four to 15 markers, with almost zero overlap, though two tests reached similar conclusions about my odds.
In the case of rheumatoid arthritis, though, the tests examined the same five markers, plus a few others, and delivered contradictory interpretations.
J. Craig Venter, chief executive of his namesake institute and of Synthetic Genomics, was a pioneer in sequencing the human genome in 2000. Though he issued recommendations to genetic testing companies four years ago to help them improve their predictions, he remains skeptical of their clinical value.
“Your results are not the least bit surprising,” he told me. “Anything short of sequencing is going to be short on accuracy — and even then, there’s almost no comprehensive data sets to compare to.”
Another source of variation was in the estimates of average risks. For example, 23andMe listed the typical odds of obesity at 59 percent, while G.T.L. listed them at 30 percent.
But the major issue, experts say, is that the causes of most common diseases remain unknown. Genes account for just 5 to 20 percent of the whole picture.
“Even if they are accurately looking at 5 percent of the attributable risk, they’ve ignored the vast majority of the other risk factors — the dark matter for genetics — because we as a scientific community haven’t yet identified those risk factors,” said Dr. Wendy Chung, an associate professor of pediatrics and medicine and the director of clinical genetics at Columbia.
Environmental factors play a role. A study published in 2007 demonstrated this link: After pregnant mice were exposed to various levels of a chemical, their genetically identical offspring were born obese and yellow or small and brown.
There are only 23 diseases that start in adulthood, can be treated, and for which highly predictive tests exist. All are rare, with hereditary breast cancer the most common. “A small percentage of people who get tested will get useful information,” Dr. Klitzman said. “But for most people, the results are not clinically useful, and they may be misleading or confusing.”
23andMe declined to comment for this article. Jim Bentley, the chief operating officer of General Genetics Corporation, which owns G.T.L., said test results should be interpreted with professional guidance: “Because of the complexity of genetic testing results and other factors that have a role in determining the long-term potential health risks a person may face, such as environmental conditions and personal health habits, G.G.C. requests its customers provide information that would allow us to send the results of our predisposition test to a physician.”
The chief medical officer of Pathway, Dr. Michael Nova, said: “Pathway Genomics is accredited by the College of American Pathologists, and accredited in accordance with the U.S. Health and Human Services’ Clinical Laboratory Improvement Amendments of 1988. As such, we are held to a higher standard for report accuracy than our unaccredited competitors.”
To be sure, my tests did provide some beneficial information. They all agreed that I lack markers associated with an increased risk of breast cancer and Alzheimer’s.
That said, they were testing for only a small fraction of the genetic risks for these diseases, not for rare genetic variants that confer much of the risk. I could still develop those diseases, of course, but I don’t have reason to pursue aggressive screenings as I age.
In June, the Supreme Court prohibited the patenting of genes that exist in nature. But the ban did not apply to DNA created in the laboratory, leaving the door open to additional patents in the field. That will encourage the industry to gather big bio-banks of people’s genomes — the more, the better.
In the wake of the F.D.A. warning, 23andMe has stopped providing data on health risks, and a class-action lawsuit alleges that the company’s results are “meaningless.” The company’s supporters have launched a petition to protest the warning.
Until and unless the predictive science improves, experts advise consumers to take most of their results with a grain of salt.
The tests “may be interesting as a kind of entertainment,” Dr. Caplan said, “but do not take them seriously yet in driving your health care or your lifestyle.”
He added: “If you want to spend money wisely to protect your health and you have a few hundred dollars, buy a scale, stand on it, and act accordingly.”
421 Comments
In the U.S., we have been using probabilities for weather forecasts for several decades. People ought to get used to them for medical testing.
(Of course, this isn't the only problem here.)
I used the 23andMe service. The results came back as expected for genetic disease risks. However, the ancestry feature initially had me as being 86% Sub-Saharan decent...later it was revised to 99.9% European (which make sense). I wasted about a month trying to determine where my African relatives were in the family tree.
I got this reply from 23andMe after I questioned my revised results.
"Hello,
Thank you for contacting the 23andMe Team. We recently experienced an issue with our Ancestry Composition feature that resulted in erroneous percentages being displayed for a subset of profiles in the database, including those for whom Ancestry Composition results had recently been computed. This issue presented most commonly with high percentages of Sub-Saharan African ancestry in the results.
Rest assured that this error was not related to your sample or raw data, and you did not receive results other than your own. The error occurred in the interpretation of your data only, and has since been corrected. No other feature of the site was affected.
We apologize for any confusion that may have resulted from this issue...results should now be corrected, and should align more closely with your known or expected ancestry.
Keep in mind that as we continue to improve and update Ancestry Composition, it is possible for your results to change....
Best Regards,
The 23andMe Team"
New reports regularly conflict with older ones, but the most common malefactor I see among them are titles and/or secondary report headlines in the medical press asserting strong correlations -- and/or even suggesting causal relationships -- when in fact there aren't any and/or the correlations are so weak as to be meaningless.
Moreover, far too much is made in the supposedly "professional" medical media of the results of research on very small populations. I regularly see articles asserting one cause-and-effect or another based on research populations of eight and ten cases. Beyond that, much of what the "scientific" media reports is based on animal -- rather than human -- studies.
Finally, it looks like most of the research into genetics is sponsored by large pharmaceutical companies that may be looking for data to support the efficacy of their medicinal products' supposed mechanisms of action on the chemistry ostensibly effected by genetic expressions.
With regard to commercial genetic testing, I'm forced to suggest, "Caveat emptor."
RG, Psy.D.
After so many years of wondering blind, I am deeply grateful to have any information about my genes, and at an affordable price. And I guess I also hold out the hope of someday finding a close "natural" family member.
I hope the FDA won't throw out the good, in favor of some impractical perfection. Even if we could analyze and know every single bit of our genetic make-up, there would still be a matter of interpreting the results in light of environmental conditions. Impossible to be completely accurate; maybe not even desirable. Instead, why not educate people about the available tools, their benefits and limits, and let the research continue. To deny people some understanding because full knowledge isn't yet available, is simply cruel - at least for folks like me.
So I spent $99 on a 23andme report! It was interesting. I didn't think it was going to predict my mortality. I'm still going to my doctors appointments. Honestly, the worst part was supplying all that saliva from a dry mouth.
When it comes to our genetic risk, we are each dealt a hand of cards and how we play that hand is everything. Some have an excellent genetic profile, but squander their good fortune on the Standard American Diet (SAD). Conversely, many who were not as lucky inherited lousy odds in the DNA game, but have made the most of the hand dealt and strive for good health.
Yes, there may be many cases in which we have in our fairly ready control environmental factors that can tip our fates from developing a condition to not developing one. But there may be any number of cases in which that just isn't so.
It's quite possible that there are people who will develop lung cancer, or die from a heart attack or stroke, almost no matter what they do, based on their genetics. And it may be that avoiding the Standard American Diet is, for certain people, effectively impossible, given their genetic dispositions.
We love to moralize about these things, and pretend that it's all in our effective control, but there's no reason to believe that this is so for many, or perhaps even most of the conditions we fear.
Complete genome sequencing as suggested in the article also does not solve the problem as we don't know the meaning of most SNPs.
Being adopted, these tests seemed like a godsend, finally knowing my genetic medical and ancestral history. I have no idea of bloodline, clueless as to family health.
Its a bit awkward telling the Dr, "I know nothing". Not helpful.
Guessing my ancestry is a long running game I would like to see firmly resolved.
A few years ago I was willing to spend a few grand, looked into the available tests and concluded its still too early. This article reconfirms.
The flaw of ancestry being that in order to deliver accuracy, the data pool for comparison needs to be much larger. Researchers estimate at least another 10 years of gathering before results are meaningful, much less pinpoint geography.
This excellent article points to a second flaw: Even with a deep dive of scanning and analysis, science still cannot stitch together the map of DNA and tell us what to expect. We aren't there yet. In the end its kind of a 'so what'.
Add to that the piecemeal analysis of low cost services as described here and it is strictly for the fun of it.
I may contract the diseases I am genetically predisposed to having before science delivers any sort of predictive accuracy. I can live with that.
But would like to know my ancestry, not who but where.
My daughter is my only known flesh and blood.
As for my generation of adoptees, we continue to live without any knowledge of our mysteries. There's something to like about that...
Why is it that a much more complex test battery is available without support?
For many disease states, early treatment or even changing behaviors; to avoid a higher risk of said disease are critical to maintaining health. Due to this I find great value in consumers having access to such testing. However, it would be foolish to make a major health decision, such as prophylactic surgical removal of ones breasts, based solely on this type of testing.
If the companies are using different "training" data, then the prediction on a new individual is subject to variation. Likewise if different companies are using different predictive models, you're guaranteed to see prediction variance. There are a lot of factors under the hood of these companies that aren't appreciated and I would expect to contribute greatly to variation in prediction.
You would only expect to see similar prediction on an individual if you A) have very similar training data, and B) use the same predictive model(s).