MEDICINE AND HEALTH:
Tarnished Science

By Henry I. Miller

Even in the most prestigious journals, peer review isn’t always the gold standard it’s supposed to be. Studies can be grievously flawed. By Henry I. Miller.

People who use science to inform their decisions on matters such as cancer prevention, climate change, and food safety know about the hierarchy of reliability. There are “facts” in press releases, newspaper articles, and blog entries, which tend to be less trustworthy than, say, information in reports from Britain ’s Royal Society or the U.S. National Institutes of Health. But if a report appears in a peer-reviewed journal, where articles are examined by independent experts before being published, it elicits instant and widespread respect all round.

But on occasion those scientific articles have been egregiously and obviously flawed, with basic rules of research violated.

Flawed articles have raised false alarms over the safety of genetically modified (GM) plants, and such reports are then splashed across the popular press. Poor scientific papers published by trustworthy journals have helped  spread the propaganda of antitechnology activists.

In 2001, a paper in the British journal Nature proved a case in point. It purported to show that genes from a pest-resistant, GM variety of corn had migrated into native corn plants in Mexico. Months before the article was even submitted for publication, colleagues of the authors had pointed out serious flaws in the methodology and results. When the article appeared, it attracted a wave of criticism from major research groups that the journal duly published. Eventually the paper was condemned by the editor in chief: “Nature has concluded that the evidence available is not sufficient to justify the publication of the original paper. ” No other research group was able to confirm the findings.

In 2000, the American journal Science published an article in which the authors said they had evaluated the ecological risks and benefits of genetically engineered plants by reviewing the scientific literature. En route to concluding that they could draw no conclusions, the authors neglected the proven benefits of GM organisms, including enhanced yields, nutritional enhancements, less use of chemical pesticides, and more no-till farming, which reduces soil erosion and chemical runoff and puts less carbon dioxide into the environment. At the same time, no detrimental effects of GM plants had, or have, been described.

Another harmful example of apparent antibiotechnology bias appeared in the British medical journal Lancet. This now infamous paper by Arpad Pusztai, then at the Rowett Research Institute in Aberdeen, Scotland, claimed to show that modifying potatoes with a gene for a substance toxic to insects damaged rats ’ immune systems and stimulated abnormal cell division in their digestive tracts. But many research groups have shown that the experiment ’s methodology was fundamentally flawed and that no conclusions about the safety of biotech foods could be drawn from it. After an extensive review into the paper ’s methods, the British Royal Society concluded: “On the basis of this paper, it is wrong to conclude that there are human health concerns with the process of GM itself, or even with the particular genes inserted into these GM potatoes. ”

The editors of the journal remonstrated. They said that despite the article’s admittedly poor methodology and the strenuous objections of the paper’s referees, they published it to make constructive progress in the debate among scientists, the media, and the general public about a highly charged issue. Unleashing it has proved anything but constructive, because its publication is frequently cited as a presumptive validation of its spurious conclusions. The rationalization by the editors makes a mockery of the peer-review process.

The most recent example of the failure of editorial and peer review occurred in a September 2007 article in a U.S. journal, Proceedings of the National Academy of Sciences (PNAS). The authors claimed to show that pollen from GM corn was injurious to certain insects in a laboratory mock-up of an aquatic ecosystem, but their conclusions are dubious, and their methodology sloppy and inadequately described. More important, the researchers reported elsewhere that they had failed to find these same effects in studies in the field, details they neglected to reveal in the PNAS article. This is a critical omission because laboratory studies are designed to mimic what happens in the real world. In other words, even if the laboratory studies had been performed correctly and carefully, positive results arguably would have been irrelevant because they do not sufficiently mimic what happens in the field.

These kinds of failures of peer review corrupt the traditional process by which scientific knowledge is obtained and reported; they are examples of what Irving Langmuir, the 1932 winner of the Nobel Prize in chemistry, called “pathological science”—the “science of things that are not so.” Pathological science inflicts irreparable harm on the reporting and archiving of scientific developments for policy makers, the media, the public, and the scientific community. Within weeks of the publication of the flawed PNAS article, for example, European Union environmental regulators had cited it as justification for banning the sale of GM corn seeds. And such spurious findings only encourage conspiracy theorists who think that wider evidence of harm from GM plants is being suppressed.

Because science is, or is supposed to be, self-correcting (a thesis is put forth, tested, and ultimately revised on the basis of new data), any misinformation conveyed to the scientific community distorts the entire process. Journals should be more meticulous about requesting reviews of research articles only from bona fide experts who do not have a known bias toward the subject; currently, many journals fail to exercise such due diligence. After qualified, unbiased reviewers have evaluated an article, they should be encouraged to ask probing, detailed questions, and the authors of the submitted article should be required to answer them satisfactorily before a paper is accepted.

Had such measures been taken in the case of the research articles described above, it is unlikely that any of them would have been published in a prominent journal. If scientists want to retain society ’s respect, they must insist that journal editors uphold the highest standards of accuracy.

This essay appeared in the Guardian (U.K.) on March 6, 2008.

Available from the Hoover Press is To America’s Health: A Proposal to Reform the Food and Drug Administration, edited by Henry I. Miller. To order, call 800.935.2882 or visit www.hooverpress.org.

Henry I. Miller, M.S., M.D., is a research fellow at the Hoover Institution, where his research focuses on public policy toward science and technology. It encompasses a number of areas, including pharmaceutical development, the new biotechnology, models for regulatory reform, and the emergence of new viral diseases.