The Inevitable Evolution of Bad Science

Sep 24, 2016

By Ed Yong

Bacteria, animals, languages, cancers: all of these things can evolve, which we know from the work of legions of scientists. You could argue that science itself also evolves.  Researchers vary in their methods and attitudes, in ways that affect their success, and they pass those traits to the students they train. Over time, the very culture of science is sculpted by natural selection—and according to Paul Smaldino and Richard McElreath, it is headed in an unenviable direction.

The problem, as others have noted, is that what is good for individual scientists is not necessarily what is good for science as a whole. A scientist’s career currently depends on publishing as many papers as possible in the most prestigious possible journals. More than any other metric, that’s what gets them prestige, grants, and jobs.

Now, imagine you’re a researcher who wants to game this system. Here’s what you do. Run many small and statistically weak studies. Tweak your methods on the fly to ensure positive results. If you get negative results, sweep them under the rug. Never try to check old results; only pursue new and exciting ones. These are not just flights of fancy. We know that such practices abound. They’re great for getting publications, but they also pollute the scientific record with results that aren’t actually true. As Richard Horton, editor of The Lancet once wrote, “No one is incentivized to be right. Instead, scientists are incentivized to be productive.”


Continue reading by clicking the name of the source below.

3 comments on “The Inevitable Evolution of Bad Science

  • @OP – Now, imagine you’re a researcher who wants to game this system. Here’s what you do. Run many small and statistically weak studies. Tweak your methods on the fly to ensure positive results. If you get negative results, sweep them under the rug. Never try to check old results; only pursue new and exciting ones.

    We also know that people who use such blatantly dishonest methods get caught out and exposed as charlatans by their peers when their work is critically examined or when others try to replicate their experiments!

    These are not just flights of fancy. We know that such practices abound.

    That sounds very much like a projected “flight of fancy” and gross exaggeration, from someone promoting an anti-science agenda, and with no idea how peer-review works in leading scientific journals!



    Report abuse

  • @Alan4discussion

    The first three paragraphs, in isolation, introduce some current challenges in science publication. If one were to stop reading there, they may conclude that the purpose of the article also ends. A biased opinion might even attribute an intention to the article. However, the source clearly goes on to describe specific statements from both sides of an argument. It also concludes with optimism for the future of scientific research, citing ideas such as transparency, developing new standards, and changing incentives.



    Report abuse

  • Mike DeMauro #2
    Sep 24, 2016 at 2:24 pm

    The first three paragraphs, in isolation, introduce some current challenges in science publication. If one were to stop reading there, they may conclude that the purpose of the article also ends.

    There are quite a few assumptions and assertions in the model presented, which do not reflect the range of quality across the various journals.
    Some subject areas are also more noted for their lack of rigour than others.

    The top journals are mindful of their reputations, so look for high standards in the studies they publish.

    As the article points out, without rigorous efforts to maintain standards, quality can slip.

    @OP link – Top journals like Nature and Science are indeed encouraging authors to be more transparent about their data and methods, while providing checklists to make it easier for editors to inspect the statistical qualities of new papers. And Nosek’s Center for Open Science has created standards for transparency, openness, and reproducibility that journals and funding agencies can sign up to, and badges for good behavior.

    We only have to look at the popular media to see the depths to which unchecked published information can descend.
    One of my concerns is that those first paragraphs can easily be misquoted or cherry picked, by those who seek to undermine the credibility of valid science in general, for political or commercial purposes.

    @OP -A scientist’s career currently depends on publishing as many papers as possible in the most prestigious possible journals. More than any other metric, that’s what gets them prestige, grants, and jobs.

    I would have thought, that “more than any other metric”, what gets them prestige, and jobs, is discoveries which have groundbreaking applications of their work in dealing with real world issues.
    Surely this is the ultimate “replication of results”, and has nothing to do with multiple publications of papers!



    Report abuse

Leave a Reply

View our comment policy.