October 2006--In this era of expensive and highly competitive science, peer-reviewed journals—and researchers—are inexorably driven toward publishing studies that showcase positive results and subsequently produce sexier headlines and more prominent recognition.
This trend, laments Hopkins Oncology Professor Scott Kern, means that publishing negative results, which ultimately can aid positive findings, has fallen through the cracks. He notes, for example, that a decade ago, molecular geneticists were a small, tight-knit group where “everybody knew everybody else and all results, including negative data, were shared in this community.”
Except in rare instances when negative findings contradict en vogue positive results or occur in clinical trials that require all effects be reported, papers with data that don’t support a hypothesis are increasingly relegated to the publishing backburner. At best, they appear briefly in discussion or methods sections; at worst, they remain trapped in a dusty notebook.
Suppressing these results can waste time, money and resources as researchers unknowingly repeat others’ negative findings. Logically then, efforts should be made to get negative data out there. After all, notes Jerry Hart, who heads Biological Chemistry, “if experiments are designed properly, then they always provide information. Even negative results become positive because you know more than when you started.”
Such thinking led Kern to assemble an online repository called the Journal of Negative Observations in Genetic Oncology (NOGO). “It basically serves as a data hub,” he explains. “People write up brief reports of negative results they found or observed, with links to the original paper, and it’s searchable by mutation or tumor type.”
But in an age of massive data generation, when even keeping up with less frequent positive findings is nearly impossible, one must ask: Is it right to share successes and failures equally? At the heart of this question, says Randy Reed, a professor of molecular biology and genetics, “is that there are 100 ways to get a negative answer, but usually only a few to get one that’s right.” He points to his own field of mouse genetics, in which knocking out the same gene in two mouse strains can yield remarkably different results. “Is a KO mouse showing no apparent effect an important negative finding or did they just use the wrong strain?”
Bjorn Olsen of Harvard Medical School took this point of quality assurance into account when he and medical student Christian Pfeffer developed their own forum in 2002: the Journal of Negative Results in Biomedicine. “Meaningful negative results that can advance science need peer review,” says Olsen. While data repositories like NOGO may be useful, especially in high-throughput fields, they face the risk of including results that are negative due to errors in experimental design or contamination among the relevant findings.
On the other hand, would researchers take the extra effort to put together and submit—or subsequently review—a negative article that may have little value? “If someone puts a lot of time in a project that ultimately fails, they don’t want to wallow in it,” says Kern. “They would rather put it behind them and move on.”
Whatever the system used to publicly display negative findings, though, other concerns can arise once the information is out there. Reed notes that some scientists fear scaring people away from potentially promising study directions by reporting negative results. “Also, sharing reagents is an obligation that accompanies publishing,” he adds. “If you report incremental negative results, must you share all those materials so others can pursue the lines of experimentation that you’ve just initiated?”
Olsen and others don’t believe increased exposure of negative findings will induce too much fright. “Most successful scientists are stubborn by nature,” Olsen says, “and they will challenge anything that they don’t believe in, positive or negative.” In this regard though, Olsen adds that journals such as his should ensure that they publish only well-documented negative studies that meet the rigors of science and not fall into the trap of several “positive” journals which sometimes consider the potential newsworthiness or controversy of findings over their experimental quality.
Ultimately, databases of negative findings may not make a huge difference. Despite the initial enthusiasm for NOGO—“I think I’ve gotten more pats on the back for that than anything else I did,” says Kern—entries from Kern’s lab or from his colleagues at Hopkins constitute the majority. Some external people have listed negative observations they read about, but no one outside Hopkins has volunteered their own. “It has become a great source of institutional memory, though,” he says.
Submissions to JNRB have also remained low, although Olsen believes they will rise now that researchers better understand what the journal wants. Still, when many scientists are coy about discussing preliminary positive findings, the only thing most negative results will likely see in the near future is more dust.
– Nick Zagorski