So you think of science (or maybe I should say “science”?) as a solid, objective, trustworthy activity? Certainly a whole lot more credible than, say, philosophy, or theology, or fortune telling?

Before I go on, let me assure you that I value science (without the scare quotes) a great deal. (Philosophy and theology, too—but not fortune telling!) But today a lot of what goes by the name of science deserves the scare quotes. And that should scare us, for a lot of reasons, because it means we’re likely to make a lot of costly mistakes under the guise of science.

Back in April, Pascal-Emmanuel Gobry wrote in The Week, “Big Science is broken,” and in May, software engineer William A. Wilson wrote a sobering article in First Things, “Scientific Regress.” His lead sentence boldly states, “The problem with science is that so much of it simply isn’t.”

By what standard? The standard that says a finding, to deserve to be called science (or at least good science), has to be replicable. That is, “experiments repeated under nearly identical conditions ought to yield approximately the same results ….”

But, Wilson writes, “until very recently, very few had bothered to check in a systematic way whether this was actually the case” with many scientific (or “scientific”) studies published in prestigious journals.

Enter the Open Science Collaboration. It sought to replicate “one hundred published psychology experiments sampled from three of the most prestigious journals in the field.” Long story short? “Of the studies that had originally reported positive results, an astonishing 65 percent failed to show statistical significance on replication, and many of the remainder showed greatly reduced effect sizes.”

Similarly, attempts to replicate pharmaceutical studies found that 75 percent failed. In cancer research, 89 percent failed. Even in physics, generally considered the most concrete and rigorous of the natural sciences, “Two of the most vaunted physics results of the past few years—the announced discovery of both cosmic inflation and gravitational waves at the BICEP2 experiment in Antarctica, and the supposed discovery of superluminal neutrinos at the Swiss-Italian border—have now been retracted, with far less fanfare than when they were first published.”

Much of the failure can be chalked up to simple human error, but as Wilson points out, a not insignificant amount comes from fraud. And the rise of post-normal science (“science”) has made that all the more prevalent as science has turned into a game of going through the motions of scientific research to promote a predetermined political agenda.

And don’t assume that peer review solves the problem. It doesn’t work—and indeed it has become part of the problem, as Wilson explains:

If peer review is good at anything, it appears to be keeping unpopular ideas from being published. Consider the finding of another (yes, another) of these replicability studies, this time from a group of cancer researchers. In addition to reaching the now unsurprising conclusion that only a dismal 11 percent of the preclinical cancer research they examined could be validated after the fact, the authors identified another horrifying pattern: The “bad” papers that failed to replicate were, on average, cited far more often than the papers that did! As the authors put it, “some non-reproducible preclinical papers had spawned an entire field, with hundreds of secondary publications that expanded on elements of the original observation, but did not actually seek to confirm or falsify its fundamental basis.”

What they do not mention is that once an entire field has been created—with careers, funding, appointments, and prestige all premised upon an experimental result which was utterly false due either to fraud or to plain bad luck—pointing this fact out is not likely to be very popular. Peer review switches from merely useless to actively harmful. It may be ineffective at keeping papers with analytic or methodological flaws from being published, but it can be deadly effective at suppressing criticism of a dominant research paradigm. Even if a critic is able to get his work published, pointing out that the house you’ve built together is situated over a chasm will not endear him to his colleagues or, more importantly, to his mentors and patrons.

For more depth on this, see John P.A. Ioannidis’s “Why Most Published Research Findings Are False.”

The upshot of all of this is that those who, like us, follow the Apostle Paul’s instruction to “test all things, hold fast what is good” (1 Thessalonians 5:21) and so are skeptical of all scientific (“scientific”) findings (“findings”), especially in relatively young and extremely complex, multidisciplinary fields like climate science (“climate science”), are science affirmers.

And those who insist that we bow and scrape before the (imaginary) overwhelming consensus of “climate scientists” are the real science deniers.

 

 

Featured image courtesy of Glen Edelson, Flickr Creative Commons