Because a pre-print comment is somewhat formal, I thought that I should separate my opinions from the main feedback.
So, I decided to put those in a blog post. You can see my pre-print review/comment here, and these are the extra comments:
General Notes / Warnings (Completely Removed from Comment):
My Nebula lcWGS results were OK for some things (like
relatedness and broad ancestry), but I found the Gencove accuracy to be
unacceptable for
specific variants (for myself).
While Nebula has changed to only provide higher coverage
sequencing, I previously submitted an FDA
MedWatch report for my own data (for the lcWGS Gencove results).
To be fair, there are also general limits to the utility of
most of the Polygenic Risk Scores that I was able to test with my own data
(with some informal notes in this
blog post). So, while true,
mentioning that I still had concerns about the percentiles that I saw from
Nebula (even with the higher coverage sequencing data) may be less relevant.
Similarly, while I want to encourage other customers to
report anything they find to MedWatch (and/or PatientsLikeMe, etc), I
also want to acknowledge my own limitations that this general warning is more
about specific issues that I found for myself.
For example, it may help to have an independent analysis with
larger sample sizes to gauge my general PRS concerns and/or be more specific in
terms of which specific PRS do or do not have clinical utility with sufficient
predictive power for the disease association.
Specific Comment #2) I think my own result might match the imputed correlation that is described (in terms of having ~90% accuracy). However, I would say that is unacceptable for making clinical decisions, especially since more accurate genotypes can be defined. It is important to be transparent and not over-estimate accuracy, so I think that part is good. I also realize that something unacceptable for individual variants can be acceptable for other applications. However, I think something about limits should be mentioned for the general audience, even if they really apply to the same Polygenic Risk Scores in higher coverage sequencing data.
I am not sure if this matters for this particular project, but I have found that it is not unusual to learn about something that may contradict an original funding goal. I have certainly noticed that it can take me a while to realize I need to question some original assumptions, but sharing those experiences is extremely valuable to the scientific community (if the conclusions then shift to helping others avoid similar mistakes). I also realize prior assumptions can be hard overlook in comments/reviews as well, and there is definitely more that I can learn. Given that you have a pre-print and there is a lot of details in supplemental information and external files, I think that is a good sign.
Specific Comment #3) In the future, I hope that this is also the sort of thing that precisionFDA, All of Us, etc. can help with. In fact, as an individual opinion, this makes we wonder if the SBIR funding mechanism might be able to help with directly providing generics through non-profits (especially for genomics diagnostics). However, I don’t think that means SBIR for-profit funding would have to be completely ended to preferentially fund non-profits, and I realize that probably can’t affect this particular paper.
If the Gencove code isn’t public, then I am not sure how you
could show others could reproduce a freeze of the code before testing
application to new samples. Nevertheless, I applaud that you provided
some code for the publication.
Specific Comment #4) There may be a way to revise the current manuscript without adding the independent (public) test data and/or the open-source alternatives. For example, I don’t think you need additional results for your effective coverage section, but I am more interested in the concordance measures. If the Gencove / STITCH / GLIMPSE / IMPUTE results are similar in terms of technical replicate concordance (for the same 1000 Genomes samples), then I think that you could skip what is described for specific comment 3) for this paper.
I also noticed that the competing interests statement was in
the past tense for the present employees (as I understand it).
Summary: I think the utility for lcWGS to cause additional genomic data types to be considered identifiable information is important (which I have in a different blog post).
Change Log:
5/6/2020 - public post
Posts
