Updated Thoughts on PatientsLikeMe

I have a previous post about PatientsLikeMe, but I importantly did not test creating an account until relatively recently.  I have been continually improving my habits in terms of taking more time to critically assess results and question prior assumptions (in addition to realizing that I may have not had the best title for that previous blog post, in retrospect), so I thought there would be value in providing an updated perspective on this free website.

I have genomics / medical data publicly available to download on my Personal Genome Project page (for hu832966) and I have what I would consider a partial electronic medical record on my PatientsLikeMe page (which I think is an excellent resource for sharing and learning about patient experiences, with the requirement that everybody who participates be completely open; however, you have to sign in with a free account to view my profile).

For those that currently don't have PatientsLikeMe accounts, I thought I should describe a few of my experiences (from the perspective of a patient):

I have taken Citalopram at doses of 20 mg and 40 mg (and 0 mg, during intervals to test the continued benefit of the medication, when my overall stress levels were lowered and/or I learned better cognitive strategies to manage stress).  While it makes quick analysis more difficult, I think being able to see the details of people's experience can be important.  For example, I thought it was interesting that my body's reaction to the medication seemed to change over time (each time I went back on the medication, I think the side effects were more subtle, even though I think the severity of my initial symptoms also gradually improved over time).  If this is in fact true, that would indicate some resistance / reaction that could not be completely captured from studying germline variants (if you are focusing on using DNA genotyping/sequencing for medication guidance), such as somatic variants, epigenetic modifications, etc.

I also like that PatientsLikeMe provides scores for both effectiveness and side-effects (and I admittedly created a PatientsLikeMe account because some plots in the "Health Communities" in 23andMe reminded me of what I had seen for PatientsLikeMe, even without previously creating a PatientsLikeMe account).

On the positive side, I have seen multiple neurologists, and I had previously not really found any of the previous migraine medication that I took to be helpful.  However, my most recent neurologist prescribed me indomethacin, and I found that to be very helpful.  I wrote a positive evaluation for that migraine treatment, and I was surprised to see that this was a relatively rare treatment for migraines.  So, if people found commonly prescribed treatments to not be helpful, I think this might be helpful in brainstorming alternatives.

I also reported 3 negative evaluations for drugs where I experienced moderate-to-severe side effects.  I noticed that severe side effects were self-reported for these drugs among 9-14% of members in the Community Reports (9% was comparable to other drugs that I checked, but the drug for which I had the most severe side effects in 2018 had a the highest severe percentage of 14% and qualitatively most frequent reports that seemed similar to by own experience).  That said, the most commonly prescribed migraine medication (which I never tried) had a reported severe side effect rate of ~20% (so, it seems to me that a self-reported "severe" side effect rate of 5-10% is normal, but 15% or 20%  with hundreds or thousands of patients may be kind of high).  That said, I want to be very careful about being too negative about something that is not my area of expertise (even though the idea of something being helpful for some people and harmful for others seems relevant for genomics research).

Going back to the topic of my anti-depressant (for anxiety or depression, depending upon the time-frame of my treatment that you are talking about), the current maximum recommended dosage of Citalopram is 40 mg (with 60 mg now being considered unsafe), and that would match my own expectation (although for slightly different reasons - I had to drink coffee instead of tea due to extra drowsiness at 40 mg, and I am currently on 20 mg instead of 40 mg).  I can also see a 2016 indication from the FDA that 20 mg is the maximum recommended dose for individuals greater than 60 years of age (so, the maximum recommended dose is currently lower for older individuals).  You can also see more information about this drug in the 1998 drug approval package from the FDA.  To be clear, I am very grateful for the availability of Citalopram and that has made a huge difference in my life, but I think this is something that may be worth discussing more (and I would probably also benefit from understanding better).

There has even been Washington Post article describing a partnership between PatientsLikeMe and the FDA to help with drug reporting (I saw this in a recent e-mail from them, but the article is actually from 2015 - still, it is good to know other people probably have at least somewhat similar thoughts).  While they didn't mention PatientsLikeMe, I think this was also related to the topic of a more recent announcement regarding patients reporting "real-world evidence."  You can also report adverse events to the FDA through MedWatch.

Update Log:

3/9/2019: original blog post
3/26/2019: changed link in 1st paragraph (and added another link in that sentence).
6/28/2019: add MedWatch link

Who was the Primary Audience for the FDA Warning to 23andMe?

NOTE: Getting Advice About Genetic Testing

The other day, I put together an informal survey about public opinion on the FDA warning to 23andMe and the subsequent class action lawsuit against 23andMe.  I posted links to this survey on Twitter, Facebook, the 23andMe Community discussion group (sorry, link only visible for 23andMe users), and in the comments for two articles describing the class action lawsuit (see specific links here and here).

Since it looks like the response rate has plateaued, I am presenting the results for everybody to see in this blog post.  By the way, I don't work for 23andMe, so I can't provide authoritative "right" answers to some of the questions.  However, I will always provide my own answer.

Question #1

Pretty straightforward.  I wanted to see how responses compared for 23andMe customers versus everybody else.  Almost all responses came from 23andMe customers.

I responded "Yes"

Question #2

Most people think 23andMe is at least as good as other companies.  Although the response rate is limited, I think the 2 individuals that thought 23andMe was "less accurate" provide some potentially interesting information that I will describe in more detail throughout the post.

I said "Comparable".  For SNP calling, I am assuming that most companies are using customized Illumina SNP arrays.  If this is the case, the sequencing quality should be practically identical, regardless of where you go.

Question #3

The vast majority of those surveyed don't think they deserve a refund (which is the basis for the class action suit).  If you're counting the answers for each question, you may notice that 2 people that didn't buy a 23andMe kit accidentally answered this question (both answering "No").  So, really only 45 23andMe customers answered "No," but I think the most important point is that only 2/47 customers expressed concern about their results.

The two individuals that said "Yes" are the same two individuals who thought 23andMe was less accurate than other direct-to-consumer genetic testing companies.  Again, I only have a small sample size, but this may indicate that FDA warning has caused a lot of concern about the quality of 23andMe results that didn't previously exist.

If this is true, this is important to keep in mind because I think it represents a misinterpretation of the FDA warning.  I think it is safe to say that the main message was something along the lines of "Hey 23and and Me - I want to learn more information about your tests, please reply to us ASAP".  This is not the same as "Hey current and existing customers - we are aware of serious problems with 23andMe, so you should not buy this bogus test".  My current theory is that confused customers are taking the later interpretation.

Moreover, this appears the be the same misconception implied by a blog post trying to recruit people for the class action lawsuit:

http://www.markankcorn.com/blog/2013/12/2/class-action-filed-against-23andme-for-fraud-and-deceptive-practices

More specifically, the blog post says "23andMe, it turns out, has been under investigation since 2008 by the FDA".  While technically this is probably true, this seems to be imply that 23andMe has been singled out for suspected wrongdoing for 5 years.  However, this is not the case.  The first news from the FDA I recall hearing was in 2010, where a warning was issued to all the major direct-to-consumer (DTC) genetic testing companies (in fact, I believe Pathway Genomics was issued that warning before 23andMe).  The issue is that these companies are providing an unprecedented service that requires thinking about regulation in new ways.  Getting FDA approval is time-consuming in normal circumstances, and the process for 23andMe will take even more effort.  This is the nature of the long-term "investigation".  The FDA warning last month relates to "510(k)s for PGS on July 2, 2012 and September 4, 2012".  We know that 23andMe was not prompt in its responses since that time.  I would consider this a foolish mistake, but 23andMe can still eventually provide a product that is officially deemed acceptable by the FDA.  In fact, I would bet this will eventually be the case.

If you're still wondering, I said "No".

Question #4

My own opinions have changed around a bit on this one.  As you might guess from the question design, I've assume the true cost to provide the test was greater than the amount charged to the customer (and I choose $100-$500).  Namely, I know 23andMe has a lot of strong financial backing, including NIH funding for their research (which I hope can also help allay concerns caused by the FDA warning: clearly, the folks in this other branch of the federal government are cool with 23andMe).

I have become less certain about the costs recently when I found out that AncestryDNA also provides a genetic test for $99 (click here to see the details from my blog post on that).  My assumption was this company has less extra funding than 23andMe.  The 23andMe array provides ~50% more SNPs than the AncestryDNA sample that I looked at, but I'm not sure how much this really makes a difference in cost for the array.  That said, 23andMe probably has a lot more work to do on the interpretation side (AncestryDNA only provides ancestry results) as well as work with surveys to conduct genomics research.  So, I would probably still give the same answer in the end, but I am less confident in my choice.

To a certain extent, these details don't really matter too much: I would consider $99 dollars to be a great deal for the genetic information alone (even without interpretation from the vendor).  It is safe to say 23andMe is not making huge profits from the collection and processing of DNA samples.

Question #6 (Yes, #6 - I'll explain)

One of my concerns is that I've seen a lot of information on the Internet that I found to be inaccurate or misleading (even from generally respected sources of information), and I was concerned this was causing an exaggeration of concern in potential or existing 23andMe customers.

This question is a follow up for a question where I asked what articles / blogs people read reading the FDA warning and/or the lawsuit.  I'm not showing those results here because they are hard to read.  There was basically just a bunch of check boxes next to hyperlinks (for sites containing both positive and negative opinions about 23andMe).  20/51 individuals didn't read any of the same links that I did, and the follow up question showed that most people didn't really care what they read on-line from these types of sources.  Neither of the individuals desiring a refund read any of those specific links.  One said "No" to this question, the other said "Maybe".  Obviously, they learned the news from somewhere (and again, we're only talking about 2 people), but I did feel better that some of the specific sources that concerned me were probably not a huge deal.

I answered "No" for this one.

Additional Results

I originally used all 10 allowed questions for the free survey, but I ended up deleting 4 of those questions.  Basically, I was also interested in learning other stuff:  What to people think the terms "true positive" or "false positive" mean (with respect to 23andMe results)?  How confident are customers in their quality of results (raw sequencing data, interpretation of results, etc.)?  What to people think the error rate is for the sequence data?  (BTW, I would use this comment / post to argue the error rate less than 0.01%)  What is the perceived difference in quality for the health versus ancestry results?  How familiar are 23andMe customers with the scientific concepts being used and the limitations to the results?  However, I ended up creating confusing questions that jumbled these issues together.

I bring this up because I was impressed by how quickly I got a response from the community of 23andMe customers.  Within hours, I had users point out that the questions couldn't be properly answered without more specific information.  In fact, I received strong worded complaints due to this problem in designing the survey.

In other words, the process of running the survey made me feel more confident in the ability of 23andMe customers to be able to handle and critique their results.  Of course, I have read studies showing this to be the case (at least most of the time), but the act of collecting the data itself provides an experience that went beyond reading statistics in a scientific publication.

Concluding Remarks

The bottom line is that I feel insulted a class action lawsuit is being sought with the assumption that all 23andMe customers deserve a refund (I assume that assumption is necessary, based upon the requested $5 million dollars).  I would never ask for this, and I strongly oppose the lawsuit.

Of course, it would be better if my survey included a larger number of customers.  Hopefully, there can be a more authoritative inquiry into this matter.  My gut says that most customers are satisfied with their 23andMe results and those that are concerned about their results have over-estimated the severity of the potential problems, and I think this limited survey supports that theory.  In the very least, I hope it encourages discussion about this topic and eases some customer concerns.

If I get significantly more responses, I will update this post.  However, I'm not expecting the final count to be much higher (and I can't actually collect more than 100 total responses).

Survey of Opinions on 23andMe FDA Warning and Lawsuit

I was surprised to see so many negative (and/or confused) responses to what has been going on with 23andMe the past week or so.  So, I would like to try and better quantify what is the true public opinion on the matter.

Click here to take survey

I've never actually done this before...I'll eventually write a post describing the results, but I don't know how long it should take to get a reasonable number of results.  I'm hoping a week, but this will definitely depend on how much people pass along the survey.

UPDATE: You can see the results of this study here: http://cdwscience.blogspot.com/2013/12/who-was-primary-audience-for-fda.html

My Take on the FDA Warning to 23andMe

NOTE (8/5/2020): As I gained more experience finding problems that were not clear until after 5-10 years of research, I looked back at this and I would not have completely agreed with earlier responses.  So, I have separated my first response (Response #1) and additional responses (Response #2).  I have also started to keep a change log, for updates after this point.

NOTE: Getting Advice About Genetic Testing

I have recently participated in a forum discussion on Biostar about the newest warning to 23andMe from the FDA.  I think my responses will be of interest to a broader audience, so I have copied them here.  If I end up contributing more to the Biostar discussion, I will update my blog post as well.

Comment: The majority of published reactions seem to miss what the FDA's major issue is: about marketing and describing the service.

Response #1: Yeah, I agree that the wording in this warning focuses mostly on marketing.

However, the FDA has previously tried to shut down DTC genomics companies (including 23andMe) and the 3rd paragraph seems to mostly focus on the accuracy of the test. The carrier status report should really be OK for diagnosis (and I think many of the specific associations mentioned in that warning are also pretty well established). Now, there are some caveats to some results like deciding how to combine independent SNP risks and explaining the difference between a mutations that guarantee onset of a disease versus modulate risk (which may only have a modest impact on risk in many cases). I personally think 23andMe does a decent job of this already, but I'm sure there can always be room for improvement.

In other words, my understanding is that the problem was primarily with direct communication with the FDA regarding technical benchmarks that would justify marketing claims (and I think the FDA is supposed to provide permission based upon this data prior to advertising). This certainly relates to communication with customers, but I think delays in formal responses to the FDA from 23andMe were the primary problem.

Response #2: In retrospect, I don't think I should have used the phrase "shut down".

I might add additional thoughts, but this was the main thing that jumped out when I re-read the blog post.

Concern: There is at least one report of an individual with an inaccurate 23andMe report (click here to view).

Response #1: Make sure to read the entire article.  Once the bug was reported, it was fixed and the report was updated.

Response #2: Errors need to be made clear to customers.  You can see some notes like this among this collection of blog posts (which includes submission of multiple FDA MedWatch reports), but I don't believe any of this was made clear to other customers.

While a lack of confidence is sometimes necessary to communicate, it does need to be communicated.

There can also be negative consequences.  For example, if the trace for the automated Sanger sequence was not checked, then sequencing error could be a false positive for a pathogenic mutation.  If this was not caught before action was taken (which could be something like an unnecessary mastectomy), then permanent damage could be caused from providing a result prematurely and/or inaccurately.

That said, I don't want to over-emphasize the potential harm, and I think helping citizens become engaged in problem solving and critical assessment would be valuable (if genomics data / hypotheses were used for that purpose).

Concern: The issue isn't the bug. The issue is that 23andMe is offering a product while making claims about how customers can use results for improved medical care. Medical professionals and should be in charge of offering medical services.

Response: I agree that communication with customers is important and some customers may not have a good sense of what it is like to be part of a research project. It is possible that this is something 23andMe needs to work on.

However, I think the best solution is not to get rid of 23andMe, but rather help improve communication regarding the confidence of results. For example, I wrote a blog post about one possible solution after a previous FDA warning was issued:

http://cdwscience.blogspot.com/2010/08/benefits-to-3-tier-system-for-dtc.html

Also, I don't think this is a typical result. For example, here is a link to my results as well an article from Lifehacker (from someone with much less experience with genomics research). I'm sure I've seen more, but these are what I could think of off the top of my head.

http://cdwscience.blogspot.com/2011/02/thoughts-on-my-23andme-results.html

http://lifehacker.com/5802559/how-to-decode-your-dna-with-personal-genomics-service-23andme

Plus, I think an unfortunate reality is that this sort of thing will happen from time to time. I think pretty much all diagnostics will suffer from some degree of false positives, false negatives, and/or human error. I know I constantly have to update the bioinformatic programs that I design (for what I would call "research grade" analysis) - especially when hunting down bugs that are only apparent when analyzing a small number of data sets.

Concern: The fact that it turned out to just be a bug is one thing, it's pretty bad but they fixed it, but even if it had turned out to be true, it's still horrific. Imagine being sent an email saying, "Hey, you're going to get progressively disabled then die young," and no further information about what the condition is and how it's going to affect you, no kind sympathetic face offering you tissues and advice and options. This guy did his research and turned out to be fine, and that's great, but how many people fall into deep depression on getting this news? How many people kill themselves? You can't give out this kind of potentially devastating life-altering news in an email.

Response #1: For some people, I agree this may not be the best way to communicate results. This is probably why 23andMe adds an additional step for viewing results like this that are not present for non-medical and non-predictive results. If you aren't prepared to view results on-line, then I would probably recommend either not getting a 23andMe profile, not viewing that portion of the results, and/or contacting a genetic counselor to review the results with you. For example, my 23andMe report indicates that I am a carrier for cystic fibrosis and they provide a link on how to talk to a genetic counselor on that page:

https://www.23andme.com/you/genetic_counseling/

That said, I think the concern overall is an over-reaction for the following reasons:

1) There have been several publications showing that most people have no problem responding to DTC genetic testing. I can't list all the publications off the top of my head, but here is a summary of one such article:

http://www.nature.com/news/2011/110112/full/news.2011.12.html

2) In general, the accuracy for the DNA sequencing portion of the tests (currently via an Illumina SNP array) is pretty good. For example, the FDA has recently approved Illumina sequencing for clinical application. I'm also pretty sure 23andMe has checked the accuracy of the array by comparing normal 23andMe clients to the results from people who participated in the exome sequencing pilot. That said, there is a difference between the interpretation for the carrier status results (which is relatively straightforward) and all of the other results, and my understanding is that failure to communicate these results to the FDA is one of the legitimate complaints from the FDA letter.

3) I think people need to be careful and critical in all cases. For example, let's say I had a wife who was also a cystic fibrosis carrier (identified via 23andMe) and we were thinking about kids. The first thing I would do is verify the result. For example, I could order a Counsyl test from a doctor (which might be a good alternative for some people instead of 23andMe, although I think you might still be viewing your results on-line) to verify that we were in fact both carriers. I wouldn't immediately run to an in vitro fertilization clinic. Plus, medical professionals can make wrong calls too. Also, to be clear: I have family members who are confirmed cystic fibrosis carriers, as determined by standard testing.  So, I think the probability of this being a false positive is very low, but I would always want to tread carefully.  This has certainly happened to me, which at least one time delayed hospitalization for a very serious infection. This is not an attack on the medical establishment: there is a reason I went to see the doctors in the first place. However, I think the actions made by this individual were spot on, regardless of whether something is FDA-approved and regardless of whether a result comes from a person or a computer: if something doesn't sound right, you should look into a second opinion, independent research, etc.

Response #3: As an update to 3), I later submitted my samples to multiple companies.  With the raw data, I would confirm that I am cystic fibrosis carrier.  However, multiple companies said that I wasn't a carrier.  So, I think having raw data and taking time to evaluate results is important.

Comment: There is now a class action lawsuit against 23andMe: http://gigaom.com/2013/12/02/23andme-hit-with-class-action-over-misleading-genetic-ads/

Response #1: That is unfortunate.

Response #2: While I hope issues can be resolved outside of court, I now agree that 23andMe (and AncestryDNA) ads can be misleading.  As one example, I had/have serious concerns of advertising Airbnb destinations (as mentioned in this blog post).

While I usually kept the previous responses, I thought this paragraph should be changed (hence the different font color).  Essentially, I think the link below is worth reading, but my impression is different.  I still think it was important to change the title (to avoid exaggerating the problem).  However, I think important points were also raised and I apologize for not sufficiently appreciating that before:

http://liorpachter.wordpress.com/2013/11/30/23andme-genotypes-are-all-wrong/ 

I'm still hoping that most conflicts can be settled out of court (if that is still possible at this point). At least this provides a list of specific claims that I hope 23andMe will directly address to customers in an official statement - at least they can reference a plethora of 3rd party experts who can generally back them up. I certainly think they made bad choices with the timing of advertising and providing terse official responses, but I don't think that should be a $5 million mistake (especially for a service that I assume is being provided below cost)

Update: I also have put together a survey on this topic. If you can fill out and/or distribute the survey, I would appreciate it!

http://cdwscience.blogspot.com/2013/12/survey-of-opinions-on-23andme-fda.html 

Change Log:

11/26/2013 - public post date

8/5/2020 - start keeping change log with updated responses

8/6/2020 - continue to add revised responses

How and Why the FDA Should Allow DTC Genetic Testing

At the beginning of this month, the FDA extended the period to submit public comments about Direct-To-Consumer (DTC) genetic testing to the Molecular and Clinical Genetics Panel of the Medical Devices  Advisory Committee (referencing docket ID FDA-2011-N-0066 at http://www.regulations.gov).  For more information on this topic, please check out this post from The Spittoon (the official blog for 23andMe).

I just submitted a comment to the FDA (which is essentially a shortened version of this blog post).  You can currently view my comment here using Google Docs, but I do not currently see the posting on http://www.regulations.gov (I will work on verifying that the comment was successfully uploaded).  In fact, there were only a few comments posted after the original 3/1/2011 deadline, and I do not see any new comments posted after the extension of the comment period that occurred on 4/1/2011.  If you have not already done so, please submit a comment before the new deadline on May 1st!

In many ways, I think DTC genetic testing companies are similar to medical websites like WebMD (which is an idea I first remember seeing in this blog post comment).  I personally think it would be a great disservice to society if websites like WebMD, Mayo Clinic, and MedlinePlus were banned because they provide medical advice to the public without consultation with a physician.  Likewise, I also think it is very important that people be able to learn about their own genetic information without having to consult a physician (although I would certainly encourage people to seek advice from medical experts if they feel the need to do so).  Although all doctors do not agree that patients should have access to DTC genetic tests, there are also some doctors that dislike WebMD.  I do not believe this is a valid reason to ban either type of medical information.

I want to emphasize that I do not oppose any sort of FDA regulation.  For example, companies that intentionally mislead people should be penalized (as one example, check out this post on My Gene Profile by Daniel MacArthur).  However, I do not think the FDA should ban companies who are transparent in their actions and are basing their analysis of published, peer-reviewed scientific research.

I think there is sufficient evidence to show that most people will have reasonable reactions to their results (for example, check out this research article in New England Journal of Medicine).  However, I think it might be helpful for the  FDA to help classify which test results clearly require medical action and which ones are "research" grade tools that connect people with findings in the medical literature.  In an earlier post, I discussed how a "3-tier system" might be able to help accomplish this.  Essentially, we currently do have "clinical" tests and "research" tests, but I think formalizing some sort of system to distinguish between such tests could be useful (especially if it helps provides a way to maintain DTC genetic testing without the need to require physicians act as a gatekeepers for this information, or if it prevents these tests from being outright banned).

In general, I think it is important for individuals to have access to a variety of opinions in order to think critically when making medical decisions.  It is not good to blindly trust any source of information - whether that information comes from a doctor, a DTC genetic testing agncy, a government regulatory agency, or a scientist (like myself).  I strongly believe that people should have access to second opinions about their genetic tests (through tools like Promethease).

I think the FDA could also potentially help improve genetic testing (for both DTC and non-DTC tests) by helping provide people access to secondary sources of information.  For example, I think it would be fine for the FDA to force companies to allow users to export their data in a standard format in order to allow people to easily get second opinions about their genetic testing results.  Strictly speaking, I don't think this is necessary - for example, there are 3rd party web apps that help users learn more about their 23andMe results (such as this Firefox app), and Promethease already helps users search for annotations from SNPedia (although there is a $2 fee if you want your results quickly).  However, I don't think it would hurt to have a standard format that applies to all genetic testing companies.

In fact, a standard format for sequence data from genetic tests could provide a useful framework for a collaboration between the FDA and NIH to fund development of of a free tool for people to analyze their genetic information.  For example, MedlinePlus is an excellent resource provided by the NIH, and I think it could be really cool of the FDA would work with the NIH to help people analyze their genetic information similar to the way MedlinePlus provides traditional medical advice   If such a collaboration were to take place, then I think it would also be fair to require genetic testing companies to provide links to this 3rd party tool (as well as other tools, if they choose to do so).  This could be helpful both in terms of helping people think more critically about their results and I think it could be a good way to fund research on how to best convey genomics research to the general public and incorporate publicly available data into a single risk assessment provided by this free, 3rd party tool.

Update (6/20/2020): I wrote this post before I started adding change log entries.  However, I added a note because my opinions have shifted somewhat since I originally wrote this blog post.  For example, you can see several FDA MedWatch reports that I have submitted within the collection of posts linked here.

Essentially, I think I have better appreciation for the harm that can be caused if a result is rushed to the public, especially if information is distributed to a large number of people (such as 10,000s or 100,000s of customers).  I still believe that situations where something partially effective that still works better than a placebo (or has non-trivial predictive power) should be thought of differently than highly effective solutions or completely ineffective solutions.  Indeed, you can see some non-genomic reports in my PatientsLikeMe post, at least one of which I also submitted as an FDA MedWatch report for side effects.

I think setting the right expectations can help, but I thought the problems that I didn’t notice before were sufficiently important that I needed to add something to this post.

I also think it is important that genomic risk calculations can be validated in independent cohorts, which makes transparency and on-going quality assessment important.  For example, I still believe that publicly available information is important, which means that you can have access to it with or without a physician.  Even if there are consent limitations that require controlled access (or prohibit carrying out the experiment in the first place), I think maximizing specialist access to raw data (with accurate documentation of data sharing) is still important.  If you look at the cystic fibrosis post, you can see that free and open feedback from a Biostars discussion helped with re-analysis of my raw data.

Should the FDA regulate direct-to-consumer genetic testing?

Last week, Walgreens reversed its decision to provide "spit kits" for Pathway Genomics' genetic tests due to a letter from the FDA requiring Pathway Genomics to either get FDA approval or explain why they are exempt from approval.  This week, CVS also made a similar decision to postpone selling of the Pathway tests.

Recently, I have noticed a number of blog posts that seemed to side with the FDA.  For example, 80beats has criticized the Pathway Genomics tests in terms of usefulness, legality, and unpredictable public response.  Genomeboy has complained that Pathway Genomics is not being very transparent in terms of explaining their analysis (especially in comparison to 23andMe).

I agree that a lack of transparency and usefulness would be serious problems for genetic tests.  As mentioned in my first blog post, there are indeed many problems with the current accuracy of genetic tests (in terms of discrepancies between companies, incorrect prediction of clearly known characteristics such as eye color, etc.).  I think it is also essential that companies provide you with your SNP data so that you can try to seek alternative opinions regarding how to interpret the genetic test.  I was also very disturbed that the CSO of Pathway Genomics claims that "[Pathway Genomics] don't feel that [they are] practicing medicine" even though they wish to sell genetic tests in a drugstores.

That said, I think FDA regulation should only be used as an absolutely last resort.  First, I think many other criticisms are not warranted.  For example, the Wall Street Journal has a nice article about how the negative public response to genetic tests has been exaggerated.  Second, I think it will be valuable for consumers to have access to objective analysis of the accuracy and usefulness of commercially available genetic tests (either conducted through public or private means), but I don't think that necessarily has to be done through FDA regulation.  For example, a large database (perhaps something like PatientsLikeMe) of genetic test results, medical history, and lifestyle changes can help provide the necessary information to consumers.  It will take a significant amount of time to throughly examine these genetic tests, and I think this analysis can be conducted much quicker (and perhaps even better) if the public has direct access to the tests.

I would be ok with some sort of warning label that the tests are not completely accurate and other analysis should taken into consideration when making medical decisions, but I think it would be too extreme to completely remove these tests from the market.