By Professor Roy Benaroch, M.D., Emory University
Medical data transparency is crucial if we want to trust that the benefits of the medication our doctor is prescribing us outweigh the risks. This means that a drug company makes all data from medical trials, good and bad, available to the public. Unfortunately, as revealed in the Tamiflu controversy, this rarely takes place.
New Evaluation Reveals Holes in Tamiflu Studies
Hoffmann-La Roche, the producer of the flu medication Tamiflu, claimed in a 2003 study that Tamiflu reduced hospital admissions by over 60 percent, and reduced serious complications of flu by two-thirds.
Tamiflu became part of the standard, recommended care for flu in many countries.
But the news about Tamiflu changed. In Japan, episodes of hallucinations and psychosis were reported—events that weren’t seen in the earlier studies. The 2000 Lancet paper didn’t report any adverse reactions at all, which in retrospect is difficult to believe.
In 2006, the US Food and Drug Administration (FDA) urged caution in using Tamiflu, warning doctors to look out for abnormal behaviors perhaps caused by the drug. In 2007, a new Roche study again reported no neuropsychiatric side effects.
But it was the Cochrane evaluation that opened a lot of eyes not only about Tamiflu specifically but about some of the pitfalls in drug development and regulation. The Cochrane collaboration is a well-respected nonprofit organization with about 11,000 members.
They objectively, and without any industry support, review data on medical therapies. Their reviews of Tamiflu up through 2008 were positive, reflecting the overall impression that Tamiflu was safe and effective. It was a simple comment on their website from a Japanese pediatrician named Keiji Hayashi that led to an investigation that continues to reverberate.
Hayashi pointed out that the big 2003 meta-analysis that was instrumental in supporting the effectiveness of Tamiflu contained only 10 studies—and of those 10, only two had actually been published as full articles in peer-reviewed, established medical journals.
The remainder had been informally presented at meetings or published only as summaries, and all 10 of these studies had been sponsored by the drug’s manufacturer.
This is a transcript from the video series The Skeptic’s Guide to Health, Medicine, and the Media. Watch it now, on Wondrium.
The Cochrane reviewers agreed that the Tamiflu data needed another look, and went back to the study authors to get their more-complete data. They then learned that in most cases the authors didn’t have the data—that information was held by Roche, since Roche, the drug company, sponsored the studies.
The Cochrane team also learned that none of the government organizations, including the US FDA, ever had access to the full data, either. Cochrane enlisted the help of the British Medical Journal and Great Britain’s Channel 4 News, who helped pressure Roche to release their data, complete and with no strings attached. American media outlets, like The Atlantic, started to voice similar concerns.
Learn more about the dark side of medical publishing
How the Press Oversimplified Tamiflu Study Results
In 2009, Hoffman-La Roche committed to making all of the Tamiflu data from all of their trials available to researchers, but it took four more years until this was done, until the Cochrane collaboration finally got all of the data they needed to independently and completely assess the evidence.
They examined 83 trials, including many that had yet been unpublished—that was, at the time, all of the randomized controlled trials that had ever been done. They published their revised review in 2014, and their conclusions were quite different from what had been said before.
Tamiflu, Cochrane concluded, did shorten the length of symptoms, by less than a day, in adults, from an average of seven to 6.3 days. But Tamiflu had no net effect on serious complications, as measured by hospitalization rates.
Tamiflu, used to treat influenza, really had only modest benefits.
However—and this part of the review was largely ignored by the media—Tamiflu was successful in preventing influenza: 55 percent overall and 80 percent effective if looking only at household members exposed to flu.
An 80 percent prevention is fairly impressive. If there was a huge outbreak in a pandemic year, reducing the spread by even close to that amount could have a big overall public health impact.
The 2014 headlines, based on this review, show how quickly the media can change their tune and how headlines can oversimplify or gloss over complicated issues.
A 2004 story from NBC News had talked about what families should stock in their homes, in case the flu were to strike. They included tissues and acetaminophen, both very good ideas, but also prescription antivirals, which they said would be effective “70-90 percent of the time.”
But keeping prescription medicines on-hand led to hoarding of medications, and headlines like this one, from 2005: “Run on Drug for Avian Flu Has Physicians Worried.” This story discussed how people were begging doctors for Tamiflu prescriptions to buy their own family stockpile. These stories reinforced the idea that Tamiflu was a crucial drug to be hoarded.
A little more extreme, a headline from 2011: “Tamiflu Dose Might Have Saved NW Girl.” This was a tragic story about a girl in Tasmania who died of influenza because she did not receive Tamiflu.
As late as January 2014, the director of the CDC in a press briefing said that Tamiflu prevents complications, prevents hospitalizations, and saves lives and that it’s not prescribed often enough.
A few months later, after the Cochrane report, stories had a different outlook, like this one from the BBC: “Millions Wasted on Flu Drug.” The Scientist magazine said, simply, “Tamiflu is a bust.”
The media pendulum, here, swung too far. The new report showed that most people treated for influenza with Tamiflu probably get only a modest benefit with about a day less of symptoms.
But, remember, it also showed that Tamiflu has some real effectiveness in prevention and that during a pandemic year, when influenza rates can soar, it might be a crucial part of our public health response.
Is it worth stockpiling? There is no clear answer. But “We don’t know” is a sentiment rarely seen in headlines.
Learn more about how to think like a skeptic when reading news in any medium
The Implications of the Tamiflu Controversy for Medical Studies
Though randomized controlled trials are the gold standard in medical studies, in the case of Tamiflu, many of the studies were not well-designed. For instance, some of the placebos included ingredients known to cause stomach upset, so in comparison, the GI symptoms occurring in the Tamiflu branches of the trial were obscured.
Documentation of complications and adverse events was often passive or selective. One very crucial measurement, the complication of pneumonia, was most often assessed based only on the patient’s self-reporting.
Imagine that in a medical study, rather than reviewing the medical records or the chest x-rays to see which patients had pneumonia, you just asked the patients themselves. Once these details came to light, it was clear that many Tamiflu studies were biased and unreliable.
Most importantly, until Cochrane got involved and insisted on getting their hands on the data, these details about the studies’ findings and shortcomings had been hidden from public view. The benefits of Tamiflu had been overstated and the harms were hidden. It took years to figure this out.
The biggest scandal here is that Roche broke no laws by withholding or slow-pedaling their data. To this day, details from clinical trials, especially those sponsored by drug companies, never see the light of day.
Though Tamiflu became, in a way, the poster child for the problem of non-disclosure, there are probably many other drugs out there with doctors making treatment decisions based on incomplete and misleading data.
The medical media deserves credit for helping hound Roche to release their data, but in retrospect, they—and the doctors, hospitals, and public health officials—were too quick to accept the manufacturer’s claims about how well the drug worked.
Learn more about why institutional changes regarding data availability can make all the difference
Ensuring More Reliable Medical Trial Data
The motto needs to be, “Show me the data”—not just the data you want me to see, but all of it.
Institutional changes are now underway to enforce the disclosure of full trial data. Europe’s drug agency will make all of the clinical trial data they use to approve drugs public, at least for all drug applications going forward. For drugs already approved, we may still be in the dark.
Several big drug companies, including Roche, have announced policies to improve data transparency. Large databases like the one maintained at alltrials.net encourage researchers to register and share information from all of their trials, not just the ones that end up published in medical journals.
Both the UK and European authorities now require this kind of registration, so at least future researchers can more easily and quickly find all of the data rather than only the data that the drug makers want us to see.
There’s not just a lot of money at stake here; it’s your health on the line. Medical journal editors, nonprofits like Cochrane, and government health authorities all have their role.
But it’s up to the health media to make sure that consumers get the truthful, complete health information they need.
Common Questions About Medical Data Transparency
Healthcare transparency refers to the practice of providing the public with clear, comprehensible information about the medical care they are receiving. This includes prices as well as potential risks and benefits of surgeries, medications, and other treatments.
Transparency is important for both healthcare providers and patients. It allows healthcare providers to measure their progress and allows patients to make educated decisions regarding their treatment.
Established in August 2013, the Physician Payments Sunshine Act ensures that all transactions between manufacturers—those who create pharmaceuticals and medical devices—and hospitals, are reported and accounted for.
Healthcare organizations that establish transparency in their practices establish trust with patients. Additionally, transparency is beneficial to the organizations as well in that it enables them to see where they can reduce costs and increase efficiency.
