“The unfortunate side-effect of this differential treatment,” writes Lindzen, “is that a self-generating consensus slows the forward progress of scientific knowledge—a situation well described by Thomas Kuhn in his book The Structure of Scientific Revolutions.”

Ross McKitrick here (PDF), John Christy and David Douglass, Roy Spencer, James Delingpole, Patrick Courrielche here, here, and here, Steve McIntyre, and I  here and here have discussed peer review failure and corruption in climate science, where high-stakes politics and business have driven it to extremes.*

It now arises that the failures occur not just in climate science but across the board, as the article “Classical Peer Review: An Empty Gun” (published in a peer-reviewed journal!), summarized and commented on here, reveals. Writes Richard Smith in his study of peer review published in Breast Cancer Research:

… almost no scientists know anything about the evidence on peer review. It is a process that is central to science – deciding which grant proposals will be funded, which papers will be published, who will be promoted, and who will receive a Nobel prize. We might thus expect that scientists, people who are trained to believe nothing until presented with evidence, would want to know all the evidence available on this important process. Yet not only do scientists know little about the evidence on peer review but most continue to believe in peer review, thinking it essential for the progress of science. Ironically, a faith based rather than an evidence based process lies at the heart of science.

Smith quotes Drummond Rennie, deputy editor of the Journal Of the American Medical Association and intellectual father of the international congresses of peer review that have been held every four years since 1989, as saying, “If peer review was a drug it would never be allowed onto the market,” and goes on to say, “Peer review would not get onto the market because we have no convincing evidence of its benefits but a lot of evidence of its flaws.”

Four years ago, Robert Higgs also discussed peer review with considerable insider’s insight:

Peer review, on which lay people place great weight, varies from being an important control, where the editors and the referees are competent and responsible, to being a complete farce, where they are not. As a rule, not surprisingly, the process operates somewhere in the middle, being more than a joke but less than the nearly flawless system of Olympian scrutiny that outsiders imagine it to be. Any journal editor who desires, for whatever reason, to reject a submission can easily do so by choosing referees he knows full well will knock it down; likewise, he can easily obtain favorable referee reports. As I have always counseled young people whose work was rejected, seemingly on improper or insufficient grounds, the system is a crap shoot. Personal vendettas, ideological conflicts, professional jealousies, methodological disagreements, sheer self-promotion, and a great deal of plain incompetence and irresponsibility are no strangers to the scientific world; indeed, that world is rife with these all-too-human attributes. In no event can peer review ensure that research is correct in its procedures or its conclusions.

Remember that the next time somebody tries to shut down debate by saying, “But is your source peer reviewed?”

Increasingly, peer review is being replaced by “peer-to-peer review,” which, I wrote early last year,

… is in principle really no different from how science has worked for centuries. Good scientists are notoriously skeptical. Given time, they test and re-test each other’s conclusions, and the hypotheses or theories of one generation are the discarded rubbish of the next.

What makes peer-to-peer review different is its rapidity, made possible by the medium of the Internet. Weblogs like Steve McIntyre’s Climate Audit, Lucia Liljegren’s The Blackboard, Jeff Id’s The Air Vent, Anthony Watts’s Watts Up With That?, and others enable rapid-fire dissection of newly published articles by people who are absolute sticklers for detail. An author who has fabricated, fudged, exaggerated, cherry picked, or suppressed data or who has used bogus statistical methods simply doesn’t have a chance. He and his article will get ripped apart like a pig in the [piranha-infested] Orinoco River.

My conclusion then seems even more fitting now:

Truth will out. As Jesus said, “there is nothing covered that will not be revealed, and hidden that will not be known” (Matthew 10:26).

The demise of peer review by the politicization and ideologization of science is sad, but its replacement by peer-to-peer review should be celebrated. Peer-to-peer review is the free market of ideas at its best.

[*The third paragraph of this article was revised July 29, 2013, and again August 28, 2017, by updating obsolete hyperlinks. An additional hyperlink was updated October 19, 2024.]

Featured Image Courtesy of Feelart/Freedigitalphotos.net