By Roy Benaroch, M.D, Emory University
One negative point of medical journals is how subscriptions, sensationalism, and success sometimes end up dictating their content. Another point to note is the increasing symbiosis of universities and predatory journals. What are some of the other negative aspects of medical journals impacting contemporary science?
The Fukushima Impact Study
One example of shoddy science was discussed in a Popular Mechanics article from 2014, titled “What Can We Do About Junk Science?”
The article discussed a 2012 study that claimed to show that there were almost 14,000 extra deaths in the United States caused by radiation fallout from the Fukushima nuclear plant disaster in Japan. That’s a huge number. Could that possibly be true?
Radiation can kill people quickly at high enough doses; but even in Japan, among workers right there at the plant, there were only six deaths, and none of these was the result of radiation exposure.
It is biologically impossible for so many deaths to be attributable so quickly to the tiny amount of radiation that could have reached the U.S. And the Popular Mechanics article pointed out how silly the assumptions of the paper were.
Flaws in the Fukushima Impact Study
The authors had used data collected from U.S. cities: recording the number of deaths in the months after Fukushima versus the same months one year previously. But there is no reason to think those changes were attributable to Fukushima—there were population shifts, different weather, changes in luck, and some cities had more or fewer deaths as usual. Furthermore, the authors counted 119 cities afterward, but only 104 cities beforehand; so, of course, the ‘after’ numbers were higher.
It was a poor study using a false premise to reach a false conclusion, and kudos to Popular Mechanics and several science-oriented bloggers for calling that out.
Learn more about new drugs on the block.
The Publication Process of Medical Journals
It’s clear that this Fukushima study was flawed, but it was nonetheless published in a legitimate, peer-reviewed scientific journal. How did that happen? The publication is what makes a study real—that is, accessible to other scientists and journalists and policy-makers, and able to have an impact on knowledge and progress. So it’s an important question to ask, who decides what gets published, and how?
UK’s The Guardian ran a thought-provoking long format article on these issues in 2017, titled “Is the Staggeringly Profitable Business of Scientific Publishing Bad for Science?” The article pointed out that publishing is big business. Elsevier, one of the biggest publishers of medical journals, collected the equivalent of over $3 billion in revenue from its approximately 2,500 medical journals in 2010, with a profit margin of 36%.
That margin is higher than Apple, Google, or Amazon. But the business model, the way companies like Elsevier work, is unique in the publishing world.
An ordinary publisher of an ordinary magazine has to pay its writers to write and has to pay its editors and fact-checkers. Medical journals, though, don’t pay the scientists who write their studies; and even the fact-checking—the so-called peer review—is usually done by scientist volunteers in their spare time.
This is a transcript from the video series The Skeptic’s Guide to Health, Medicine, and the Media. Watch it now, on Wondrium.
The Negative Side of Medical Journals
Money is made by selling subscriptions, the more journals published, the more money they make. So the system favors an ever-expanding quantity of medical journals, all of which have to fill their pages. And some of that filler, as one can imagine, might not always represent top-quality research.
There’s another factor at play here too. Journals need to sell subscriptions, so editors favor studies that will be noticed: new things, and exciting things, and positive results. Studies that repeat experiments or refute the effectiveness of a new drug are very important, but they’re not thought to be as effective for selling subscriptions.
Learn more about alternative medicine in the news.
Open Access Medical Journals
Unfortunately, a recent trend in publishing is further muddying the water, making it easier for poor studies to get published and get noticed. It started innocently enough, with so-called ‘open access’ journals that appeared first in the 1990s. They eliminated costly subscription fees and allowed anyone to view the articles; but in return, many charged the authors a fee to publish—usually a modest fee, to cover administrative expenses.
There are many excellent journals out there that follow this model, adhering to good publishing standards including robust peer review and editorial standards. But not all of them follow good scientific discipline, and that’s a growing problem.
Predatory Medical Journals
Jeffrey Beall, a professor at the University of Colorado, coined the term ‘predatory journal’ for these kinds of schemes.
Dr. Beall kept a list of these journals and publishers for several years. He found that between 2011 and 2017 the list of suspect publishers grew from 18 to more than 1,100, and the number of predatory journals reached thousands.
A Finnish study found that in 2014 alone, almost half a million studies were published, all with minimal editorial oversight and review, in these kinds of journals.
In one famous case mentioned in the New Yorker article, an article consisting of the single phrase “Take Me Off your F-ing Mailing List” repeated over and over for 10 pages was accepted for publication in the International Journal of Advanced Computer Technology.
Learn more about the media and weight loss.
Symbiosis of Universities and Predatory Journals
Scientists and universities share the blame for this problem. The New York Times explained this well in their 2017 article “Many Academics Are Eager to Publish in Worthless Journals”. Though some researchers are employed by private companies and industry, many work for colleges and universities, and those institutions insist that their faculty publish their research.
The more publications, the better. One may have heard the phrase ‘publish or perish’: it means that if faculty members don’t get their work in print, they won’t keep their job, and they won’t be able to get another job, either. College faculty members have lived under this kind of pressure for years.
But now there’s an all-too-easy way of circumventing established standards and publishing whatever you want for a fee. And those fees are often paid by the universities themselves.
It’s true that at least some academics have probably been duped into submitting to these journals. But it’s also true that many researchers and universities know exactly what’s going on here. As The New York Times puts it, “The relationship is less predator and prey … than a new and ugly symbiosis.”
Company-funded Research in Medical Journals
One of the most well-known examples of conflict-of-interests is illustrated by the story of a brand of pain medication called Vioxx, which became one of the world’s top-selling medications. NPR recapped the controversy in their article, “Timeline: The Rise and Fall of Vioxx,” published in 2007.
In 2000, a paper was published in the prestigious New England Journal of Medicine, showing that Vioxx caused fewer GI side effects than other similar medications. The heavy marketing that was based on the paper’s findings helped propel sales, turning Vioxx into a blockbuster drug.
But over the next few years, other studies made it clear that Vioxx was also increasing the risk of heart attacks and stroke. Worse, data that would have made that clear wasn’t included in the New England Journal paper. Merck, the manufacturer, was accused of hiding that data with the collusion of some of the study’s authors. Subsequently, it was found that the authors were being paid by Merck.
The Consequences of the Vioxx Case
In September 2004, Vioxx was withdrawn from the market in the U.S. But by then, the NPR article says, 88,000 Americans had suffered heart attacks from taking Vioxx, of which 38,000 died.
In 2005, the editor-in-chief of the New England Journal of Medicine told NPR that the journal had been, “hoodwinked by Merck”.
Not all relationships between scientists and industry are nefarious, and not all industry-sponsored studies are bad science. But medical companies do have a legitimate interest in funding research to develop new treatments and technologies. Where does one draw that line?
It’s not a simple question, but clearly the most important protection is for academic journals, scientists, and industry to be more transparent—to make all relationships public, and to share all raw data collected so anyone can independently review how a study was done.
Positive Study Pressure in Medical Journals
There’s one more example of ‘bad science’, and it may be the most significant shortcoming of the current system of medical publication. It’s a pervasive bias, and it affects every stakeholder in the chain: from the scientists designing and performing studies, to nonprofits and government agencies that provide grants to pay for studies, to the editors who choose which studies to publish, to the journalists who choose which published studies make headlines.
Everyone along this chain has a bias to perform, publish, and publicize what are called positive studies—studies that show something worked rather than reporting on interventions that fell flat.
Lack of Coverage for Negative Studies
The website HealthNewsReview reviewed the problem in their essay, “Null but Not Void: Why Health Journalists Need to Stop Ignoring Negative Studies”.
They looked at the broad coverage of initial studies that seemed to show something works. But subsequent, much better experimental studies that include placebos and controls, don’t get the same kind of coverage. And if they’re negative, they’re likely to not get any coverage at all.
There is a strong international effort underway to ensure that all studies: positive and negative, complete or incomplete, published or not, are registered on websites that allow public access.
But for now, people are only likely to hear about studies with positive findings. And if what people hear is not the complete picture, that’s bad science.
Common Questions about the Negative Side of Scientific Journals
Yes, open access journals of good quality are generally reliable because they follow a rigorous peer-reviewed process.
Many authors publish in predatory journals because they are either duped or forced to publish their content in such journals.
The negative side of science journals includes subscription, sensationalism, and success pressure along with the other conflicts-of-interest, such as company-funded studies.
Lack of peer reviews and editing services are some of the defining aspects of a predatory journal. They also charge a publishing fee, which is considered as their main source of revenue.
