How and Why the FDA Should Allow DTC Genetic Testing

At the beginning of this month, the FDA extended the period to submit public comments about Direct-To-Consumer (DTC) genetic testing to the Molecular and Clinical Genetics Panel of the Medical Devices  Advisory Committee (referencing docket ID FDA-2011-N-0066 at http://www.regulations.gov).  For more information on this topic, please check out this post from The Spittoon (the official blog for 23andMe).

I just submitted a comment to the FDA (which is essentially a shortened version of this blog post).  You can currently view my comment here using Google Docs, but I do not currently see the posting on http://www.regulations.gov (I will work on verifying that the comment was successfully uploaded).  In fact, there were only a few comments posted after the original 3/1/2011 deadline, and I do not see any new comments posted after the extension of the comment period that occurred on 4/1/2011.  If you have not already done so, please submit a comment before the new deadline on May 1st!

In many ways, I think DTC genetic testing companies are similar to medical websites like WebMD (which is an idea I first remember seeing in this blog post comment).  I personally think it would be a great disservice to society if websites like WebMD, Mayo Clinic, and MedlinePlus were banned because they provide medical advice to the public without consultation with a physician.  Likewise, I also think it is very important that people be able to learn about their own genetic information without having to consult a physician (although I would certainly encourage people to seek advice from medical experts if they feel the need to do so).  Although all doctors do not agree that patients should have access to DTC genetic tests, there are also some doctors that dislike WebMD.  I do not believe this is a valid reason to ban either type of medical information.

I want to emphasize that I do not oppose any sort of FDA regulation.  For example, companies that intentionally mislead people should be penalized (as one example, check out this post on My Gene Profile by Daniel MacArthur).  However, I do not think the FDA should ban companies who are transparent in their actions and are basing their analysis of published, peer-reviewed scientific research.

I think there is sufficient evidence to show that most people will have reasonable reactions to their results (for example, check out this research article in New England Journal of Medicine).  However, I think it might be helpful for the  FDA to help classify which test results clearly require medical action and which ones are "research" grade tools that connect people with findings in the medical literature.  In an earlier post, I discussed how a "3-tier system" might be able to help accomplish this.  Essentially, we currently do have "clinical" tests and "research" tests, but I think formalizing some sort of system to distinguish between such tests could be useful (especially if it helps provides a way to maintain DTC genetic testing without the need to require physicians act as a gatekeepers for this information, or if it prevents these tests from being outright banned).

In general, I think it is important for individuals to have access to a variety of opinions in order to think critically when making medical decisions.  It is not good to blindly trust any source of information - whether that information comes from a doctor, a DTC genetic testing agncy, a government regulatory agency, or a scientist (like myself).  I strongly believe that people should have access to second opinions about their genetic tests (through tools like Promethease).

I think the FDA could also potentially help improve genetic testing (for both DTC and non-DTC tests) by helping provide people access to secondary sources of information.  For example, I think it would be fine for the FDA to force companies to allow users to export their data in a standard format in order to allow people to easily get second opinions about their genetic testing results.  Strictly speaking, I don't think this is necessary - for example, there are 3rd party web apps that help users learn more about their 23andMe results (such as this Firefox app), and Promethease already helps users search for annotations from SNPedia (although there is a $2 fee if you want your results quickly).  However, I don't think it would hurt to have a standard format that applies to all genetic testing companies.

In fact, a standard format for sequence data from genetic tests could provide a useful framework for a collaboration between the FDA and NIH to fund development of of a free tool for people to analyze their genetic information.  For example, MedlinePlus is an excellent resource provided by the NIH, and I think it could be really cool of the FDA would work with the NIH to help people analyze their genetic information similar to the way MedlinePlus provides traditional medical advice   If such a collaboration were to take place, then I think it would also be fair to require genetic testing companies to provide links to this 3rd party tool (as well as other tools, if they choose to do so).  This could be helpful both in terms of helping people think more critically about their results and I think it could be a good way to fund research on how to best convey genomics research to the general public and incorporate publicly available data into a single risk assessment provided by this free, 3rd party tool.

Update (6/20/2020): I wrote this post before I started adding change log entries.  However, I added a note because my opinions have shifted somewhat since I originally wrote this blog post.  For example, you can see several FDA MedWatch reports that I have submitted within the collection of posts linked here.

Essentially, I think I have better appreciation for the harm that can be caused if a result is rushed to the public, especially if information is distributed to a large number of people (such as 10,000s or 100,000s of customers).  I still believe that situations where something partially effective that still works better than a placebo (or has non-trivial predictive power) should be thought of differently than highly effective solutions or completely ineffective solutions.  Indeed, you can see some non-genomic reports in my PatientsLikeMe post, at least one of which I also submitted as an FDA MedWatch report for side effects.

I think setting the right expectations can help, but I thought the problems that I didn’t notice before were sufficiently important that I needed to add something to this post.

I also think it is important that genomic risk calculations can be validated in independent cohorts, which makes transparency and on-going quality assessment important.  For example, I still believe that publicly available information is important, which means that you can have access to it with or without a physician.  Even if there are consent limitations that require controlled access (or prohibit carrying out the experiment in the first place), I think maximizing specialist access to raw data (with accurate documentation of data sharing) is still important.  If you look at the cystic fibrosis post, you can see that free and open feedback from a Biostars discussion helped with re-analysis of my raw data.

Article Review: Epigenetic suppression of the TGF-beta pathway revealed by transcriptome profiling in ovarian cancer

In this paper, Matsumura et al. develop a method to identify methylated genes in ovarian cancer patients using gene expression data from roughly 40 ovarian cancer cell lines and 20 cultured primary tumor samples.  The authors posit that this method provides a unique opportunity to study pathways affected by methylation because it directly examines gene expression.

My overall thoughts on this paper:

Pros:

• The study produced a relatively large amount of data, which is now available in GEO
• The study utilized a large amount of publicly available data, providing a very useful list of citations for anyone interested in doing bioinformatics analysis on ovarian cancer (especially those interested in methylation).
• The authors utilize useful open-source tools for pathway analysis (namely GATHER and the specialized binary regression method)

Cons:

• I think it is more likely that methylation directly suppresses EMT-related genes (such as those involved with cell adhesion) rather than repressing the TGF-beta pathway (which then regulates EMT genes).
• Unlike in other cancers, patients with methylated genes do not show a worse prognosis.  In fact, I wouldn't be surprised if patents with methylated genes had a slightly better prognosis because methylation suppresses genes associated with the epithelial-mesenchymal transition (which is associated with a progression to a more aggressive cancer).  This hypothesis is also supported by the stromal response data shown in Figure S9.

I think one of the most useful tools discussed in this paper is GATHER, which is very fast and has a simple user interface.  GATHER provides enrichment analysis for information from various databases, such as Gene Ontology, KEGG Pathways, TRANSFAC, and MEDLINE.  More detailed information about the data mined in GATHER can be found in the associated paper by Chang and Nevins.

In fact, GATHER was immediately useful in helping interpret the results of this study.  For example, I used GATHER to check the enrichment for the list of 378 methylated genes described in this paper.  This revealed that the TGF-beta signaling pathway was not the most significantly enriched pathway in the gene list, and the TGF-beta signaling pathway actually had the smallest number of representative genes in the methylated gene list (out of the significantly enriched pathways).  GATHER was also useful for studying the enrichment of pathways in the more conservative "methyl cluster" gene list (which showed a weaker association with the TGF-beta pathway and a stronger association with other pathways, such as the focal adhesion genes).  These are some of the reasons that I believe the methylation directly suppresses EMT-related genes in these ovarian cancer patients (rather than acting through the TGF-beta pathway).

Another useful open-source tool described in the paper is the binary regression method used to define the TGF-beta gene signature.  The binary regression method is especially useful for biologists without a lot of coding experience because it has MATLAB GUI with a simple, user-friendly interface (and version 2.0 is even better than the original code).  In addition to defining gene and pathway signatures, the Bild lab is also currently using this binary regression algorithm to predict drug sensitivity from patient samples.

That said, there are probably a few things I should warn potential users about before giving this product my complete stamp of approval.  Although I have played around this tool a little bit (with encouraging results), I haven't had a chance to use it as much as the relatively common R packages for SVMs (in the e1071 package) and classification trees (in the tree package).  Therefore, I can't really comment about the practical limitations of this algorithm.

I was also a little bit nervous when I saw that Anil Potti (who I mentioned in my previous blog post) was one of the authors on the original Nature paper by Bild et al. for the binary regression method.  However, Potti wasn't involved with the early framework for this method (described by West et al.), and a retraction request for one of the retracted Potti papers states "although we believe that the underlying approach to developing predictive signatures is valid, a corruption of several validation data sets precludes conclusions regarding these signatures."  Therefore, I don't think Anil Potti had any negative influence on the binary regression method.

Overall, I found this paper to be useful and informative, and I would recommend it for anyone interested in microarray analysis.

Retractions in PubMed

For those who don't know, PubMed lists retractions (in addition to the standard stuff like articles, editorials, etc.).  The details regarding how PubMed decides when to flag retractions are provided here.

With retractions on the rise, I think PubMed retraction listings can play a useful role in helping hold authors accountable for publishing misleading formation.  For example, I hope NIH reviewers check PubMed to watch out for PIs with tainted records.

However, I noticed an inconsistency today that made me curious about how these retractions are annotated.

For example, take a look of these two Anil Potti papers that have been retracted:

This is how retractions should look:

However, this other paper doesn't have the retraction designation, even though there is already a separate PubMed entry for this paper's retraction::

There is a 2007 erratum mentioned for the New England Journal of Medicine paper but no retraction flag (as could be seen clearly for the Nature Medicine paper).

If anyone can provide additional information on this topic, then I would certainly appreciate it.