Another sign that too many healthcare unicorns are decoupled from the rock-solid fundamental reality–that they work. Healthcare unicorns–those startups valued over $1 bn–are unicorns because they have patents, processes, or a line of business that has immense potential to be profitable. The standard in healthcare, unlike other tech, is the peer-reviewed study. Is this process or device effective based upon the study? Does this drug looks like it will work? Is this study validating, encouraging? Peer-reviewed research takes place before a drug or device goes into clinical trials — a precursor. It ensures a certain level of disclosure, validation, and transparency at an early stage.

Instead, these unicorns largely rely on ‘stealth research’–a term coined by Dr. John P.A. Ioannidis, the co-director of the Meta-Research Innovation Center at Stanford University (METRICS). He summed it up in his latest peer-reviewed paper, “Stealth research: lack of peer-reviewed evidence from healthcare unicorns” (co-authored with Ioana A. Cristea and Eli M. Cahan), published in the European Journal of Clinical Investigation 28 Jan: 

In 2014, one of us (JPAI) wrote a viewpoint article coining the term “stealth research” for touted biomedical innovation happening outside the peer-reviewed literature in a confusing mix of “possibly brilliant ideas, aggressive corporate announcements, and mass media hype.”

The term ‘stealth research’ was prompted to the author by the practices of Theranos–ironically, a company that started and was funded in the Stanford nexus. By the time Dr. Ioannidis’ viewpoint paper was published in JAMA in Feb 2015, Theranos had ballooned to a $9 bn valuation. His paper was the first to question Theranos’ science–and Theranos aggressively pushed back against Dr. Ioannidis, including their general counsel attempting to convince the author to recant his own writing. Three years later, we know the outcome.

This latest study concludes that there is a real dearth of peer-reviewed research among healthcare unicorns–and that it’s detrimental. It measured whether these unicorns published peer-reviewed articles and whether they publish highly-cited (in other publications) articles; compared them against companies with lower valuations; and whether founders or board members themselves impacted the scientific literature through their own citations.

The meta-study looked at 18 current and 29 exited healthcare unicorns. Highlights:

• Two companies–23andMe and Adaptive Biotechnologies published almost half of all unicorn papers–196 combined
• Three unicorns (Outcome Health, GuaHao and Oscar Health) had no published papers, and two more (Clover Health, Zocdoc) had published just one
• Seven of the exited unicorns had zero to one papers
• In fact, ‘there was a negative, non-statistically significant association between company valuation and number of published or highly-cited papers’

As our Readers know, Outcome Health had a little problem around artificially inflated advertising placement wrapped in health ed and placed in doctors’ offices [TTA 29 Jan 18]. Oscar and Clover Health are insurers. Zocdoc…well, we know their business model is to get as many doctors to sign up in their scheduling app and pay as much as possible. But it’s the drug and device companies that are especially worrisome in a stealth research model. The paper points out among other examples StemCentrx, bought for $10.2 bn in 2016 by AbbVie for its Rova T targeted antibody drug for cancer treatment, was halted at Phase III because it was not effective. Acerta Pharma, also focused on cancer treatments, was bought by AstraZeneca for $7.3 bn; two years ago, AstraZeneca had to withdraw the Acerta data and admit that Acerta falsified preclinical data for its drug.

The conclusions are that healthcare unicorns contribute minimally to relevant, high-impact published research, and that greater scrutiny by the scientific community through peer-reviewed research is needed to ensure credibility for the underlying work by these startups. “There is no need for numerous papers. Discrete pivotal, high-impact articles would suffice.”

This Editor returns to #5 on Rock Health’s Bubble Meter: high valuations decoupled from fundamentals. Based on this, the lack of publishing represents risk–to investors and to patients who would benefit from better vetted treatments. To these companies, however, the risk is in having their technology or researched poached–as well as the investment in time and money research represents.

The study authors point out several ways to minimize the risk, including collaborating with academic centers in research, validation without disclosing all technical details, secure patents, and contributing their technology to other research. A higher-risk way is to “withhold significant publications until successful validation from agencies such as the Food and Drug Administration (FDA) or the European Medicines Agency (EMA)” but usually investors won’t wait that long. ‘Stealth Research’ paper, TechCrunch review Hat tip to David Albert, MD, of AliveCor via Twitter