It is often observed that scientific publications suffer from a "publication bias" against the publication of studies that generate negative results. A study purporting to show a link between a given chemical and given health problem is more likely to be published than a study that finds no correlation. Similarly, a study purporting to show that a given drug helps a given medical condition is more likely to be published than a study that finds the drug is no more effective than a placebo. The result, some claim, is a subtle bias in the scientific literature. But this may start to change.
In today's WSJ, Sharon Begley reports (link for subscribers) on the rise of publications that specialize in publishing studies with "negative" findings.
guardians of scientific probity are fighting back. A handful of journals that publish only negative results are gaining traction, and new ones are on the drawing boards."You hear stories about negative studies getting stuck in a file drawer, but rigorous analyses also support the suspicion that journals are biased in favor of positive studies," says David Lehrer of the University of Helsinki, who is spearheading the new Journal of Spurious Correlations.
"Positive" means those showing that some intervention had an effect, that some gene is linked to a disease -- or, more broadly, that one thing is connected to another in a way that can't be explained by random chance. A 1999 analysis found that the percentage of positive studies in some fields routinely tops 90%. That is statistically implausible, suggesting that negative results are being deep-sixed. As a result, "what we read in the journals may bear only the slightest resemblance" to reality, concluded Lee Sigelman of George Washington University. . . .
. . . studies that dispute connections between a gene and a disease are among the most important negative results in biomedicine. They undercut the simplistic idea that genes inevitably cause some condition, and show instead that how a gene acts depends on the so-called genetic background -- all of your DNA -- which affects how individual genes are activated and quieted. But you seldom see such negative results in top journals.
Hence, Dr. Olsen's journal, which is full of studies disputing reported links between gene variations and disease. The Sod1 gene and inherited forms of Lou Gehrig's disease? Probably not. MTHFR and the age at which Huntington disease strikes? Uh-uh. PINK-1 and late-onset Parkinson's disease? No.
Hopefully, each of these reports kept researchers, including those at drug companies, from wasting time looking for ways to repair the consequences of the supposed genetic association. But it isn't clear that any would have been published without the new journal.
The problem of detrmining efficacy of drugs is manifold, but one element is that the drug companies that fund the studies will often not allow the researchers to publish negative results. Hence, any negative results from these studies (which are the most common studies made) are rare. As a practicing physician I have to rely on networking and the opinions of those physicians who work for other companies. Not the best way to decide what medications work best (but not the worst either as you get a fairly good assesment from pharmacology lists as to what works and what doesn't.)
This is not a matter of no news is no news. There is a built in bias from the researcher/drug company basis that dominates clinical studies now a days and that is on top of the editorial bias to not print negative results (plus journals profit from the drug companies via journal advertising.) A journal that publishes negative results is very welcome and may encourage academics and researchers to pay more attention to those results.
Perhaps the efforts that Professor Adler cites are a legitimate first step.
MD south of fly over country
MnZ
The water flows in both directions. The shear volume increases, but to what effect? The computer can be likened to the chain saw. Time was it took some effort to cut down a tree with an ax or cross cut saw. The result was some thought was put into the decision as to whether to cut down this tree. Not so now. Witness what I have just done as an examlple.
MD south of fly over country
Carson, a renowned nature author and a former marine biologist with the U.S. Fish and Wildlife Service, was uniquely equipped to create so startling and inflammatory a book.
Silent Spring took Carson four years to complete. It meticulously described how DDT entered the food chain and accumulated in the fatty tissues of animals, including human beings, and caused cancer and genetic damage.
One of the landmark books of the 20th century, Silent Spring's message resonates loudly today, even several decades after its publication.
Her careful preparation, however, had paid off. Anticipating the reaction of the chemical industry, she had compiled Silent Spring as one would a lawyer's brief, with no fewer than 55 pages of notes and a list of experts who had read and approved the manuscript.
From Wikipedia on Silent Spring.
The public is more likely to get wind of such negative reports through the MSM.
So maybe it's not a problem with what scientists are publishing but rather the MSM reporting correctly on both sides and trying to provide some balance. Maybe because the MSM wants to sell magazines, newspapers and television programs that the journalists are biased as to what sells and what does not. Sensationalized stories about how Y died because of exposure to X sells much better to the American public that X had nothing to do with Y's death.
This also helps avoid unnecessarily replicating work; if you know that your proposed experiment has been tried before and didn't generate positive results, you can move on to something else.
In 16 years of directly working for Big Pharma sponsoring studies and 20 years of actually doing such studies while in private practice never once have I asked or been asked not to publish any result. AAMOF, each protocol specifically states the decision of whether or not to publish lies with the investigator.
It is just most investigators find that publishing negative results is not very emotionally satisfying and certainly won't engender awe from your colleagues.
Updated: 12:08 a.m. PT Sept 14, 2006
WASHINGTON - The Federal Communications Commission ordered its staff to destroy all copies of a draft study that suggested greater concentration of media ownership would hurt local TV news coverage, a former lawyer at the agency says.
Contrast this to the practice of medicine. A patient comes in a reports symptoms. You come up with the "most promising" thing to look for, and you order a medical test. If the test shows the problem, the lab contacts you immediately and you contact the patient immediately and get them back in to start treatment. On the other hand, if your first guess is wrong and the test is negative, somehow the results go nowhere. The patient is completely forgotten, and unless he/she goes through the maze of gatekeepers to make another appointment, the entire diagnostic process is over. So if you are a patient, you had better find the doctor who guesses right on the first try.
Also, the whole tradeoff between the test itself and the probability of it being right is gone -- you have to go for the highest probability if you only have one shot. Imagine a symptom where there is a 20% chance that the problem is some disease that can be detected or ruled out with a $10 blood test, and an 80% chance that it is a disease that can only be detected or ruled out with a $1000 test with a 1% chance of significant morbidity/mortality. In a medical world where you pay attention to negative results and automatically keep looking for the answer when you get one, you would do the $10 test first and only go on to the $1000 test when the $10 test is negative. But in the "one chance" world you have to start with the $1000 test because you have to put your money on the 80% probability.