Personal Thoughts on Collaboration and Long-Term Project Planning: Post-Publication Review

I have another post more broadly describing the importance of comments / corrections that I have self-imposed on my papers, as well as thoughts about the science-wide error rate.

However, those are not all from work I did in a shared resource at City of Hope.  So, I thought I should summarize a subset of those points here:

• COHCAP comment #1: correction of minor typos (now upgraded as a formal corrigendum) 
• COHCAP comment #2: my personal opinions emphasizing the following points
• "City of Hope" should not have been used in the algorithm name
• I've more recently gained better appreciation for the need to have testing of methods for every paper (so, I mention that readers should not consider the best COHCAP results to be completely automated).  Given that COHCAP stands for "City of Hope CpG Island Analysis Pipeline" this is relevant to my discussions of "templates" versus "pipelines"
• COHCAP comment #3: while the Nucleic Acids Research editors were very helpful in encouraging me to look more closely at a discrepancy in the listing of the machine for processing the 450k array, they declined to post the comment because it was ultimately determined to be an error in the GEO entry rather than the Supplemental Materials for the COHCAP paper.
• I mention this in a little greater detail on Google Sites; however, I was able to confirm that the HiScanSQ (not the BeadArray) was used to process the samples because i) the BeadArray is not capable of processing a 450k array and ii) City of Hope never owned a BeadArray.
• 2nd Author Correction: Table #2 was wrong (duplication of table #1, although the table description was correct)
• 2nd Author Comment #1: Use of the phrase "silhouette plot" was not precise
• While this could potentially be an example of a concern for a bioinformatician within a biology lab, I worked on this paper when I was in the COH Bioinformatics Core.  So, I think the most important lesson is to develop habits where you stop whenever you encounter something you don't know, and set a pace (and total number of projects) where you expect to have to take some time to learn more about what you see in the literature (and how to ask the right / best questions to collaborators that are likely also busy working on multiple projects).
• 2nd Author Comment #2: Use of the phrase "silhouette plot" was also used in another paper, which was published before this 2nd author paper (even though this project was started first)
• I think this is important in terms of better appreciating the interdependence of labs supported by the same staff member (although I have started try and have acknowledgements for templates, and making notes in follow-up analysis whenever code is copied between labs prior to publication).
• Middle-Author Papers
• It is important that I am fair to everybody (regardless of whether they are a collaborator).  However, I also realize this is a sensitive issue that requires some additional internal communication.
• So, I have reduced the amount of details for these examples.  While I think there has been at least 1 correction that was initiated more than a year ago, I am (slowly) continuing to follow-up whenever something is or was not correct.
• Sorting through the details for corrections is like managing the correct workload for new projects.  If I try to figure out what exactly happened with too many papers at once, I will be more likely to make mistakes.  So, at any given time, I try to focus more on ~3 issues that I know about.
• In other words, I will be honest if asked about any errors (or potential errors).  However, if I have the advantage of being able to have discussions with people who I know better, then I think it is probably wise to focus on that as much as possible.
• I am willing to add a link to notes about middle-author papers.  However, if it is possible to wait until everything on that list has been corrected, I think that may be preferable.
• So far, I don't think that I caused most of the middle-author paper errors, but I made some mistakes for middle author papers.  So, I provide a couple examples omitting some specific details below:
• GEO Sample Label Update: Since GEO doesn't have a change log, I thought I should mention there was one prostate cancer project that I helped prepare for a GEO upload whose GEO labels were not ideal (even though the patient IDs for sample pairing for the sample were correct, the samples should have been called "sample" rather than "patient," and that has been corrected).  This was not a huge problem, but most other GEO corrections are due to me not knowing about the machine (so, they were errors from somebody that I didn't catch due to a gap in my knowledge).  So, to be fair, I thought I needed to mention this because I was the one who accidentally created the error (rather than passing along somebody else's error).
• GEO Machine / Base Calling Methods Update: There was at least one submission where machine and methods needed to be updated, both of which involved at least some previous misunderstanding on my part.

If I were to give advice to my previous self, I would say it is important for the project lead to understand the full project (and plan to spend a substantial amount of time revising and critically assessing your results).  If there is something that you don't understand, do everything you can do discuss with the other authors prior to paper submission.  After all, you will likely have to give at least a partial explanation to people asking about your project, such as face-to-face discussions where co-authors may not present.  It is also important to capture the the full amount of work required for a paper (including post-publication support).

You don't necessarily have to be a project lead to need to plan for an appropriate workload, although taking responsibility for a paper is much more difficult if you aren't a project lead (if you caused the mistake, then somebody else may experience more severe consequences for your mistake).

I think it is also important to emphasize personal limits (and the solution it provides).  If your optimum workload is 5 projects and you work on 10 projects, then you are going to encounter difficulties.  However, I think it can then help if you take your time and gain a better intuition about what you don't know (and therefore what you either need to spend more time on or possibly focus less on overall).  I admittedly still have to figure out exactly what produces the best work-life balance, and I think you have to wait to notice some of the accumulation of follow-up requests and/or post-publication support / review.  However, I think I have gotten a better feel for what that "optimal" day is like: I just have to figure out how to consistently have that each day (on the scale of years).  In other words, if you are feeling overwhelmed, then I would recommend focusing on previous positive experiences as hope that you can improve by decreasing your responsibility / workload.  I also needed to learn to recognize and manage stress better (sometimes with medication).

I think a lot of what I described above can also just be simple mistakes.  For example, I can tell that I make more mistakes if I work overtime on a regular basis or if I haven't been well-rested.  While I didn't exactly cause all of the errors that I described above, I think it is necessary for me to take responsibility whenever I was first author (or equivalent).  If I can describe myself as precisely as possible (which I realize is still a work in progress), then I hope that can also help others as well (for every level of collaboration in putting together a paper).

P.S. There were 2 general points (previously under the "middle author" section) that I think may be better to move to another blog.  I have already move that content (and I will provide links here when public).  However, in the meantime, I would say those fall into the categories of i) what is the best way to correct minor errors (which you can see in this ResearchGate discussion) and ii) explain the need and estimate of time required to provide data and code needed for a result to be reproducible.

Update Log:

7/26/2019 - public post date
7/27/2019 - revise concluding paragraph
7/29/2019 - move majority of concluding paragraph back to a draft; try to be more conservative / clear with commentary
7/31/2019 - add link to COHCAP corrigendum
8/1/2019 - mention there will need to be additional corrections
8/5/2019 - minor changes (+ add back in concluding paragraph, followed by additional trimming/revision)
8/6/2019 - minor changes
8/13/2019 - mention GEO update
9/19/2019 - mention data deposit and code sharing
9/20/2019 - expand middle-author section; minor change
9/21/2019 - minor change
9/28/2019 - add experiences learned from IRB / patient consent process
9/29/2019 - fix typos; reword recent changes
10/02/2019 - add Yapeng link
10/15/2019 - add note for gene length calculation, as well as another link to blog post (with some separated content in this post)
11/1/2019 - mention ChIP-Seq issue
1/28/2020 - add intermediate set of ChIP-Seq notes
4/4/2020 - minor changes + reduce middle author content + move general points
4/24/2020 - add link to ResearchGate discussion

9/7/2020 - minor change (removing some specific information)

12/16/2020 - add another middle-author example without any details (shifting from specific to general)

Personal Experiences with Comments and Corrections on Peer-Reviewed Papers

So far, I have at least 5 publications with examples of comments and/or corrections:

• Coding fRNA Comment (Georgia Tech project, Published While at Princeton).
• It is important to remember that anything you publish has a certain level of permanence (and you can be contacted about a publication 10+ years later).
• On the positive site, I think it is worth mentioning that your own desire to work towards helping people and being the best possible scientist is important: for example, peer review can help if you do everything you can before submission, but I recently provided more public data and re-analysis that I think improves the overall message for this paper (from my own initiative; for example, please see this GitHub page, with PDFs for text and comment figures).
• So, I truly believe peer review can help, but I think personal responsibility and transparency are at least as important.
• Corrigendum for 2-Word Typo in BAC Review (UMDNJ, Post-Princeton, Pre-COHBIC).
• More important than the correction, I think it should be emphasized that 6 months of working in a lab (especially without ever doing the experimental protocol emphasized) is not enough time to justify writing a review.
• As 1st and corresponding author, I have issued two comments (and a corrigendum) for the COHCAP paper describing a method for analysis of DNA methylation data (City of Hope Bioinformatics Core).
• There was also a third comment that NAR decided not to publish (regarding an error with the machine used in GEO, which is described in more detail on my Google Sites page and briefly mentioned in the related blog post).
• As a 2nd equal contribution author, there was a correction regarding the 2nd table, as well as 2 comments related to imprecise use of the term "silhouette plot" (City of Hope Bioinformatics Core)  
• [initiated and completed corrections to middle-author papers and deposited datasets] (City of Hope Integrative Genomics Core, Post-Michigan).
• You can see one example in this PubPeer comment (which is provided because the references were not provided with the correct methods in the formal correction).

I trimmed down the details above because I think the formatting on my Google Sites page is a little better, and I listed specifics for the details of the City of Hope papers in a separate post.  So, given that only two other papers were pre-COH, I thought it may be better to shift this post more towards the higher-level discussion.

We all have other factors that will contribute to the total amount of time on a project (such as allocating some time for personal life), and I would usually expect work responsibilities to increase over time.  For example, if you are scrambling to complete your work as a graduate student, you may want to be cautious about setting goals for jobs that would have an even greater amount of responsibility.

Some people may be afraid of pointing out similar issues in previous papers.  While possibly somewhat counter-intuitive, I think this can help build trust in the associated papers / researchers: if researchers are not transparent in their actions and overall experience with a set of results, that can contribute to public distrust (and development of bad habits that can become worse over time).  Plus, if research is an on-going process of small steps towards an eventual solution, readers should expect each paper to acknowledge some limitations and/or unsolved problems (and a fair representation of results should help in identifying the most important areas for future research).

One relatively well-known example of the impossibility of being 100% accurate in all predictions is that Nobel Laureate Linus Pauling had a PNAS paper proposing a triple-helix structure for DNA.  There was even a Retraction Watch blog post that brought up the issue of whether this paper should be retracted.  I don't believe anyone currently cites that paper with the believe that DNA has a triple-helix (rather than a double-helix) structure.  However, taking time to correct and address mistakes needs to be taken into consideration for project management, and my point is that I am trying to encourage more self-regulation of corrections (since there are papers in relatively high impact journals whose main conclusion is wrong and they haven't been retracted or corrected).

While it harder to pass judgments on other people's work, I hope that I can be a good example for other people to identify issues with their own previous work.  For example, one counter-argument to the claim that most scientific arguments are wrong is the Jager and Leek 2014 paper where Figure 4 shows a science-wide FDR closer to 15%.  In one sense, this is good (15% is certainly better than 50% or 95%), but I think the correction / retraction rate is probably noticeably less than 15% (so, I think more scientists need to be correcting previous papers).  From my own record (of 1st author or equivalent papers), my correction rate is currently a little higher than that Jager and Leek estimate (3/8, or 37.5%) but my retraction rate is currently lower (0%).  I am not saying I will never have a retraction (or additional corrections).  In fact, I think there probably will be at least a couple additional corrections (among my total publication record).  However, that is my own personal estimate, and I would like to contribute to having discussions to try and reduce this correction rate for future studies.

I believe being open to discussion can cause you to temporarily lean towards agreement, as you try to see things from the other person's perspective (even if you eventually become more confident in your earlier claim).  So, even if a reviewer/editor considers a paper acceptable to publish or a grant is OK to fund (within a relatively short period of review process), the post-publication review is a very important (and I think making funded grants and/or grant submissions public and available for comment may also have value).

I also think that having less formal discussions (on Biostars, blogs, pre-prints, etc.) can help the peer-reviewed version of an article to be more accurate (if people actively comment in a location that is easy to check, reviewers take public comments into consideration, and/or journals use public comments to select reviewers that will provide the most fair assessment).  For multiple platforms, the Disqus comment system provides a centralized way to look for commentary for at least one peer-reviewed and at least one pre-print system.  While not linked directly from the paper, PubPeer also provides an independent commentary on journal articles.  I also have some examples of comments on both those mediums on this blog post.

While not the primary purpose, I think Twitter can also be useful for peer view.  For example, consider the contribution of a Twitter discussion to this Disqus comment.  Likewise, I found out about this article about the limits to peer review from Twitter.

I also have a set of blog posts summarizing experiences that describe the need for correction / qualification of results provided to the public for genomic products (although I think catching errors in papers, or better yet pre-prints, is really the preferable solution).  While maybe having something like the Disqus system for individual results (kind of like ClinVar, GET-Evidence, SNPedia, etc.) may have some advantages, people can currently give feedback in mediums like PatientsLikeMe (where I have described my experiences here) and the FDA MedWatch.

Update Log:

2/2019 - I would like to thank John Storey's tweet to reminding me of Jeff's SWFDR publication (in a draft, prior to the public post)
7/26/2019 - public post date
8/1/2019 - remove middle author link after realizing that there will be additional middle author corrections; add COHCAP corrigendum link; add PubPeer link based upon this tweet.
8/3/2019 - add Twitter links
8/6/2019 - switch link to updated genomics summary
1/16/2020 - add link to Disqus / PubPeer comment list
4/4/2020 - minor changes
7/11/2020 - minor changes

3/20/2022 - add Oncotarget RNA-Seq comment

Speculative Opinion: Possible Advantages to Directly Providing Generics via Non-Profits

I believe there is a lot more I should learn more about this topic, and I have never been directly involved in a clinical trial.

Nevertheless, these are my current thoughts about the possibility of what might be interesting about providing having generics directly enter the clinic/market through non-profit organizations (admittedly largely influenced by my experiences in genomics, which may be less relevant for some other applications):

Possible Advantages to Patients / Physicians:

• [data sharing / diagnostic transparency] Maximize public / accessible information available in order to help specialists make "best guesses" about how to proceed with available information
• I don't believe that sale of access to raw genetic data should be allowed
• Specialists / physicians should have access to maximal information to help guide decision making process
• I think it would be nice if some information was completely public, such as population-level data from Color Genomics (even though re-processing data can probably change some variant calls, and this company isn't a non-profit).
• In general, I think it is important not to place too much emphasis on any one study.  As an example of how that could skew a true estimate of risk, I think there is a useful barplot in this paper.  While over-fitting is not always the explanation, that can be a factor and I think I have a figure in this blog post that I hope can help explain that concern.
• I am most familiar with this in the context of genomics (which would be for research or diagnostic purposes).  However, I have submitted several FDA MedWatch reports (again, mostly for diagnostics), and there is still a need for surveillance of therapeutics after they have entered the market.
• [data availability for patient autonomy] Making sure patients have access to all data generated from their samples
• Having access to your raw data should also help you be capable of getting specialized interpretation as a second opinion.
• I also think self-reporting (with the ability of the patient to provide raw data) may help with regulation (or at least setting realistic expectations about efficacy / side-effects).
• I think it may help if there was more judicious use of advertising.
• Namely, I worry that some advertisements can give a false sense of confidence in the interpretation of results.  For example, I posted this draft a little early because of 23andMe's marketing of travel destinations based upon ancestry, which I don't approve of (although I support other overall goals for 23andMe).
• That said, I think it can be useful when digital advertisements allow you to comment on them, kind of like a mini self-reporting system.
• If we are talking about a therapy (rather than a diagnostic), I would expect this should also decrease costs (and is what I most commonly think of when I hear the word "generic").  Otherwise, I am mostly talking about experience with the exchange of information, often dependent upon sequencing/genotyping from another company (like an Illumina sequencer) that frequently makes use of open-source software (or analysis where unnecessarily complexity may sometimes even cause problems).

If this makes production via non-profit preferable, then perhaps a penalty for not meeting the above requirements could be an organization could risk losing it's non-profit status.  Otherwise, I am primarily concerned that the above conditions are met (at least in genomics), and I am just curious if being a non-profit might help in sustainable accomplishing that goal (although I lack knowledge on many of the accounting and legal details, and I don't have experience running a non-profit or for-profit organization).

Possible Advantages to Providers?

• Assuming expectations are defined clearly and appropriately, participation in "on-going research" may improve understanding (and forgiveness) when there are many unknowns (and possibility even limits to what can be known with high-confidence in the immediate future)?
• I called this "Decreased liability?" in an earlier version of this post, but I have gotten feedback that makes me question whether this is precisely what I want to describe.
• If I understand things correctly (and it is possible to show that precisely defining all costs to society is difficult), it seems like forgoing royalties / extra profits in exchange for limited liability (kind of like open-source software, as I understand it) could be appealing in certain situations.
• Strictly speaking, I see a warning of limited liability within the 23andMe Terms of Service (if you actually read through it).  However, I also know that I am entitled to $40 off purchasing another kit, because of the KCC settlement.  So, I would expect actually enforcing limited liability would be easier for a non-profit (if their profits were limited to begin with, it is harder to get extra money from them).
• So, even though I believe the concept of limited liability applies in other circumstances, I think public opinion of the organization is important in terms of being patient and understanding when difficulties are encountered.
• Decreased or lack of taxes paid by non-profit?
• I think part of the point of having a non-profit is making the primary focus something other than money.  However, I think this link describes some financial advantages and disadvantages to starting a non-profit.
• There was one person who raised concerns that non-products can't produce products (at least if I understood them correctly).  While I admit that I don't fully understand the tax law, I think connections to research, education, and/or "public goods" qualify for the examples that I am thinking of.  So, I can't tell if any rules need to be changed, but I found some summaries on-line that make me think things may currently be OK (such as here and here).
• At least from my end, this page says what I thought of when I was saying something should be offered by a non-profit: "Charitable nonprofits typically have these elements:  1) a mission that focuses on activities that benefit society and whose goal is not primarily for profit, 2) public ownership where no person owns shares of the corporation or interests in its property, 3) income that must never be distributed to any owners but recycled back into the nonprofit corporation's public benefit mission and activities....In contrast, a for-profit business seeks to generate income for its founders and employees. Profits, made by sales of products or services, measure the success of for-profit companies and those profits are shared with owners, employees, and shareholders."

I also originally had a bullet point for "If profits are limited, what about refunds?".  However, I decided to place less emphasis on that point after additional feedback.  For example, I recently purchased an upgrade from 23andMe (for their V5 chip, from their V3 chip).  I noticed that I had to acknowledge that the purchase was non-refundable when I purchased the upgrade.  If it is possible (and/or tactful) for the company to provide refunds, then I think there are disadvantages to this style of not providing refunds.  However, this also made me think twice about how such an interaction would look if you were hesitant to give a refund because your profits were limited (and you have things like salary caps).  Most importantly, both non-profits and for-profits have to make sure they are not compromising safety (or unfairly representing their product).

While it is not the only reason why I think something should be provided from a non-profit, I think one characteristic of something that might need to be directly offered by a non-profit is something where there is a need to make sure the experts are in the habit of publicly announcing limitations (and mistakes) on a fairly regular basis.  In other words, if you can get an accurate estimate of a reasonable success rate, you can look more closely at situations where the success rate that either is exceptionally low or exceptionally high (although I would expect gradual improvement over time).

Also, to be fair, I think of "ownership" to be different when you talk about "owning" a pet versus "owning" a product to sell.  However, I think the concept of responsibility for the former is important, and it is also definitely possible that there are misconceptions in my understanding about the ways to provide something through a for-profit organization.

If it doesn't exist already, perhaps there can be some sort of foundation whose goal is to fund diagnostics / therapies that start as generics (without a patent)? If immediately offered as generic, perhaps there could be a non-profit donation suggestion at pharmacy or doctor's office (to a foundation that helps develop medical applications without patents)? Or, if this is not quite the right idea, perhaps another possible option that could be up for discussion could be early development in non-profit could translate into decreased time to become a generic (so, even if the non-profit is not directly providing the product with limited profit margins, the contribution of non-profit can still decrease costs to society).  This relates in part to an earlier post on obligations to publicly funded research, but I believe my current point is a little different.

There is precedent for the polio vaccine not having a patent, but my understanding that came at a great financial cost to the March of Dimes (and that is why more treatments don't enter the market without patents, even though the fundraising strategy was targeted to a large number of individuals that were already on tight budgets).

Genomics Data and Diagnostics

In "The Language of Life" Francis Collins describes the discovery of the CFTR gene.  After describing the invalidation of gene patients for Myriad, he mentions "my own laboratory and that of Lap-Chee Tsui insisted that the discovery of the CF gene, in 1989, be available on a nonexclusive basis to any laboratory that was interested in offering testing" (page 112) as well as saying "I donated all of my own patent royalties from the CF gene discovery to the Cystic Fibrosis Foundation" (page 113).

My understanding is the greatest barrier to having products frequently start out as generics is the cost of conducting the clinical trial.  I need to be careful because I don't have any first-hand experience with clinical trails, but are some possible ideas that I thought might be worth throwing out as ideas:

1. Allow data sharing to help with providing information to conduct clinical trails.  For example, lets say the infrastructure from a project like All of Us allows people to share raw data from all diagnostics (and electronic medical records), as well as archived blood draws and urine samples.  Now, let's say you have a diagnostic that you want to compare to previously available options.  If the government has access to the previous tests, the original samples, and the ability to test your new diagnostic, maybe use of that information can be combined with an agreement to provide your diagnostic as a generic (with understanding that continued surveillance also serves as an additional type of validation) is a fair trade-off?
• I'm not sure if this changes how we think of clinical trails, but I think participants should also be allowed to provide notes over the long-term (after you would usually think of the trial as ending).  This would kind of be like post-publication review for papers, and self-reporting in a system like PatientsLikeMe (which I talk about more in another post).
• Side effects are already monitored for drugs on the market
2. Define a status for something that can be more easily tested by other scientists if passes safety requirements (Phase I?) but not efficacy requirements?  I guess this would be kind of like a "generic supplement," but it should probably have a little different name.

I also believe that all participants need to have access to their own data (including the ability to look up papers that use their data for publication), but I realize that this doesn't necessarily have to be part of a clinical trail because I have accessed patient genomics data from archived samples and donors/subjects (for which I think the rules are a little different).  Nevertheless, I think it is important and relevant to the points that I am making about patients having access to their raw data.

For some personalized treatments, I would guess you might even have difficulties getting a large enough sample size to get beyond the "experimental" status (equivalent to not being able to complete the clinical trial?). Plus, if some drugs have 6-figure price tags (or even 7-figure price tags), maybe some people would even consider getting a plane ticket to see a specialist for an "experimental" trial / treatment.

Role of the FDA

From what I can read on-line, I believe there is some interest in the FDA helping with generic production, and this NYT article mentions "[the FDA] which has vowed to give priority to companies that want to make generics in markets for which there is little competition", in the context of a hospital producing drugs.  According to this reference, "80 percent of all drugs prescribed are generic, and generic drugs are chosen 94 percent of the time when they are available."

Perhaps it is a bit of a side note, but I was also playing around with the FDA NDC Database (which is an Text / Excel file that you can download and sort).  For example, I could tell my Indomethacin was produced by Camber Pharmaceuticals by one pharmacy (NDC # 31722-543), and my Citalopram from another pharmacy was produced by Aurobindo Pharma Limited (even though Camber Pharmaceuticals also manufactures Citalopram, and Aurobindo also produces Indomethacin Extended-Release, according to the NDC Database).  I thought it was interesting to see how many companies produce the same generic and how many generics are produced by each company.  At least to some extent, this seems kind of like how there may be similar topics studies by labs in different institutes across the world.  So, maybe there can even be some discussions about how there can be both sharing information for the public good as well as independent assessments of a product from different organizations (whether that be a lab in a non-profit or a company specializing in generics).

I also noticed that the FDA has a grant for "complex" generics, but I believe that is for current drugs that are off-patent but there were extra challenges with production that make offering a generic version more difficult.  Nevertheless, it is evidence that there is some belief that academic and non-profit institutes may be able to help bring generics to the market more quickly.

Personal Experience / Open-Source Bioinformatics Software

I believe that I need to work on fewer projects more in-depth.  I wonder if there might be value in having a system for independence that would allow PIs to do the same (with increased responsibility/credit/blame at the level of the individual lab).  If something entered the market as a generic (possibly from a non-profit), perhaps the same individuals can be involved with both development and production of the generic.

Also, for my job as a Bioinformatics Specialist, I mostly use open-source software (but I sometimes use commercial software or software that is only freely available to non-profits/academics).  In particular, I think it is very important to have access to multiple freely available programs, and the topic of limits to precision in genomics methods (at least in the research context) is something I touch on in my post about emphasizing genomics for "hypothesis generation" (at least in the research context).

Concluding Thoughts

Even if is not used in clinical trails (which, as far as I know, was not part of the original plan), I think All of US matches some of what I am describing as a generic from a non-profit (even though it isn't called a "generic," it is a government operation, and free sequencing is not currently guaranteed after sample collection).  Nevertheless, non-profit (or academic) Direct-to-Consumer options that I think more people should know more about include Genes for Good (free genotyping), American Gut (can still be ordered from Indiegogo?), the UC-Davis Veterinary Genetics Lab, and I am excited to learn more about others.  I think this may also be in a similar vein to DIYbio clubs (for example, I believe Biocurious provides a chance to do MiSeq sequencing).  Cores (like where I work) also kind of do this (for labs), but I can tell that I need to work on fewer projects more in-depth (so, I think there would need to be some changes before adopting a "core" model for producing generics).

Finally, I want to make clear that this is something that I would like to gradually learn more about, but that is probably more on the scale of 5-10 years.  That is generally what I am trying to indicate when I add "Speculative Opinion" to a blog post title.  So, I very much welcome feedback, but my ability to have extended discussions on the topic may be limited.

The only things that I feel strongly about in the immediate future is not reversing the Supreme Court decision to not allow genes to be patented, and the limits to predictive power for some genomics methods (such as the concerns I expressed about the 23andMe ancestry results towards the beginning of this post, and how I don't believe it would be appropriate to encourage travel destinations to specific countries).

Change Log:

5/22/2019 - original post date
-I should probably give some amount of credit for the idea of emphasizing decreased health care costs to Ragan Robertson (for his answer to my SABPA/COH Entrepreneur Forum question about generics and providing something in a non-profit versus commercial setting).  However, his answer was admittedly more focused on mentioning how generics could be used for different "off-label" applications after they have entered the clinic (as well as connecting this to decreased health care costs).
5/23/2019 - update some information, after Twitter discussion
5/24/2019 - trim out 1st paragraph
5/25/2019 - move open-source software paragraph towards end.  Also, lots of editing for the overall post.
5/26/2019 - remove sentence with placeholder for shared resources post that is currently only a draft.  Add link to $2.1 million drug treatment tweet (with every interesting comments)

6/1/2019 - remove the word "their" from 23andMe travel sentence
6/27/2019 - update content in response to discussion with family member.  For example, I don't think I was making clear that I was primarily concerned about data sharing / transparency and continuing to not allow genetic testing / information to be patented, at least in the field of genomics (and I am curious if being a non-profit can play a helpful role if those requirements are met).
6/28/2019 - revise explanation for the previous change log entry
6/29/2019 - bring up tax details
7/13/2019 - add explanation for "Speculative Opinion"
7/30/2019 - add comments for Francis Collin's CFTR gene discovery
11/3/2019 - add "speculative opinion" tag
5/6/2020 - minor changes

8/22/2020 - add a couple additional links

10/4/2020 - add FDA "complex" generic grant link + minor changes + add section headers

Updated Thoughts on PatientsLikeMe

I have a previous post about PatientsLikeMe, but I importantly did not test creating an account until relatively recently.  I have been continually improving my habits in terms of taking more time to critically assess results and question prior assumptions (in addition to realizing that I may have not had the best title for that previous blog post, in retrospect), so I thought there would be value in providing an updated perspective on this free website.

I have genomics / medical data publicly available to download on my Personal Genome Project page (for hu832966) and I have what I would consider a partial electronic medical record on my PatientsLikeMe page (which I think is an excellent resource for sharing and learning about patient experiences, with the requirement that everybody who participates be completely open; however, you have to sign in with a free account to view my profile).

For those that currently don't have PatientsLikeMe accounts, I thought I should describe a few of my experiences (from the perspective of a patient):

I have taken Citalopram at doses of 20 mg and 40 mg (and 0 mg, during intervals to test the continued benefit of the medication, when my overall stress levels were lowered and/or I learned better cognitive strategies to manage stress).  While it makes quick analysis more difficult, I think being able to see the details of people's experience can be important.  For example, I thought it was interesting that my body's reaction to the medication seemed to change over time (each time I went back on the medication, I think the side effects were more subtle, even though I think the severity of my initial symptoms also gradually improved over time).  If this is in fact true, that would indicate some resistance / reaction that could not be completely captured from studying germline variants (if you are focusing on using DNA genotyping/sequencing for medication guidance), such as somatic variants, epigenetic modifications, etc.

I also like that PatientsLikeMe provides scores for both effectiveness and side-effects (and I admittedly created a PatientsLikeMe account because some plots in the "Health Communities" in 23andMe reminded me of what I had seen for PatientsLikeMe, even without previously creating a PatientsLikeMe account).

On the positive side, I have seen multiple neurologists, and I had previously not really found any of the previous migraine medication that I took to be helpful.  However, my most recent neurologist prescribed me indomethacin, and I found that to be very helpful.  I wrote a positive evaluation for that migraine treatment, and I was surprised to see that this was a relatively rare treatment for migraines.  So, if people found commonly prescribed treatments to not be helpful, I think this might be helpful in brainstorming alternatives.

I also reported 3 negative evaluations for drugs where I experienced moderate-to-severe side effects.  I noticed that severe side effects were self-reported for these drugs among 9-14% of members in the Community Reports (9% was comparable to other drugs that I checked, but the drug for which I had the most severe side effects in 2018 had a the highest severe percentage of 14% and qualitatively most frequent reports that seemed similar to by own experience).  That said, the most commonly prescribed migraine medication (which I never tried) had a reported severe side effect rate of ~20% (so, it seems to me that a self-reported "severe" side effect rate of 5-10% is normal, but 15% or 20%  with hundreds or thousands of patients may be kind of high).  That said, I want to be very careful about being too negative about something that is not my area of expertise (even though the idea of something being helpful for some people and harmful for others seems relevant for genomics research).

Going back to the topic of my anti-depressant (for anxiety or depression, depending upon the time-frame of my treatment that you are talking about), the current maximum recommended dosage of Citalopram is 40 mg (with 60 mg now being considered unsafe), and that would match my own expectation (although for slightly different reasons - I had to drink coffee instead of tea due to extra drowsiness at 40 mg, and I am currently on 20 mg instead of 40 mg).  I can also see a 2016 indication from the FDA that 20 mg is the maximum recommended dose for individuals greater than 60 years of age (so, the maximum recommended dose is currently lower for older individuals).  You can also see more information about this drug in the 1998 drug approval package from the FDA.  To be clear, I am very grateful for the availability of Citalopram and that has made a huge difference in my life, but I think this is something that may be worth discussing more (and I would probably also benefit from understanding better).

There has even been Washington Post article describing a partnership between PatientsLikeMe and the FDA to help with drug reporting (I saw this in a recent e-mail from them, but the article is actually from 2015 - still, it is good to know other people probably have at least somewhat similar thoughts).  While they didn't mention PatientsLikeMe, I think this was also related to the topic of a more recent announcement regarding patients reporting "real-world evidence."  You can also report adverse events to the FDA through MedWatch.

Update Log:

3/9/2019: original blog post
3/26/2019: changed link in 1st paragraph (and added another link in that sentence).
6/28/2019: add MedWatch link