The Journal of Spurious Correlations

Qualitative and Quantitative Results in the Social Sciences

 
 

Home  |  News  |  Introduction  |  Manifesto  |  People  |  Research  | Participate  |  Submit   |  Contact

 

 

Excerpt of the article:  "New journals bet 'negative results' save time, money" by Sharon Begley, published in the Wall Street Journal on September 15, 2006


(...)

... publication bias, the tendency of scientists to report findings that support some point (...) but to bury examples (...) that undercut it. It has existed for years, most seriously in the failure to publish studies that cast doubt on the safety or efficacy of new drugs.

Now, guardians of scientific probity are fighting back. A handful of journals that publish only negative results are gaining traction, and new ones are on the drawing boards.

"You hear stories about negative studies getting stuck in a file drawer, but rigorous analyses also support the suspicion that journals are biased in favor of positive studies," says David Lehrer of the University of Helsinki, who is spearheading the new Journal of Spurious Correlations.

"Positive" means those showing that some intervention had an effect, that some gene is linked to a disease -- or, more broadly, that one thing is connected to another in a way that can't be explained by random chance. A 1999 analysis found that the percentage of positive studies in some fields routinely tops 90 percent. That is statistically implausible, suggesting that negative results are being deep-sixed. As a result, "what we read in the journals may bear only the slightest resemblance" to reality, concluded Lee Sigelman of George Washington University.

(...)

Keeping a lid on negative results wastes time and money. In the 1980s, experiments claimed that an antibody called Rap-5 latches onto a cancer-related protein called Ras, exclusively. Scientists using Rap-5 then reported the presence of Ras in all sorts of human tumors, notes Scott Kern of Johns Hopkins University. That suggested that Ras is behind many cancers.

Oops. The antibody actually grabs other molecules, too. What scientists thought was Ras alone was a stew of compounds. In part because the glitch was published in obscure journals, researchers continued to use Rap-5 and reach erroneous conclusions, says Dr. Kern.

"If the negative results had been published earlier, scientists would have saved a lot of time and money," adds Bjorn Olsen of Harvard Medical School, a founding editor, with Christian Pfeffer, of the Journal of Negative Results in Biomedicine.

(...)

That may sound like the set-up for a joke, but studies that dispute connections between a gene and a disease are among the most important negative results in biomedicine. They undercut the simplistic idea that genes inevitably cause some condition, and show instead that how a gene acts depends on the so-called genetic background -- all of your DNA -- which affects how individual genes are activated and quieted. But you seldom see such negative results in top journals.

Hence, Dr. Olsen's journal, which is full of studies disputing reported links between gene variations and disease. The Sod1 gene and inherited forms of Lou Gehrig's disease? Probably not. MTHFR and the age at which Huntington disease strikes? Uh-uh. PINK-1 and late-onset Parkinson's disease? No.

(...)

Questionable correlations between a gene and cancer are the bread-and-butter of NOGO, the Journal of Negative Observations in Genetic Oncology, which Dr. Kern edits. "Fully half (of discoveries) of novel mutations in tumors, we found, were not confirmed in the subsequent literature," he says. "You expect to see follow-ups if the claims held up, so the fact that we didn't casts doubt on the original claim. But that wasn't explicitly reported."

(...)

 

 

Home  |  News  |  Introduction  |  Manifesto  |  People  |  Research  | Participate  |  Submit   |  Contact