Harbingers of the replication crisis

I am going to post a series of tweets by James Heathers (who if you are not following on Twitter, you are really missing out) highlighting a series of passages dating back over half a century. They serve as warnings that, had they been heeded, would have left many of my peers in my corner of the sciences feeling a lot different about the soundness of the work we cite in our own research and teach to students:

Thesis: 'methodological crisis' in science is NOT the sudden realisation of a problem. It is a well characterised problem which benefited hugely in recognition by a change in how scientists communicate and collaborate. Central issues were outlined clearly before 1970.

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Sterling, 1959. Publication bias, the file drawer problem, and the cult of significance. pic.twitter.com/N8sOnSNrUi

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Cohen, 1962. Social scientific studies are, in general, substantially underpowered. pic.twitter.com/wVwCws1gNu

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Forscher, 1963. The pursuit of publication rather than the pursuit of reliable results. pic.twitter.com/6m5IJMaS0J

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Platt, 1964. Topic-hopping, weak theory, and the role of induction over time. pic.twitter.com/zZUxaNtfxS

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Meehl, 1967. The role of the 'cute', surprising, or counter-intuitive results as an eventual outcome for capitalizing on chance. pic.twitter.com/XrManBcdNH

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Lykken, 1968. The weakness of statistical significance in isolation, the need for replication, the central importance of methods, and a whole lot more. pic.twitter.com/e5UgZepmft

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Does all of the above form a coherent body of work that people read at the time? No idea. Probably not.

But - these papers do address, straightforwardly and in better prose than we're allowed to write now, the heart of issues that a lot of people feel blindsided by at present.

— 🏴James Heathers🏴 (@jamesheathers) October 27, 2018

Let's not only heed those who tried to warn us over a half century ago, but heed those who are warning us now. I will not be around in a half century, but I will be around long enough to suss out if we're going to be actually progressing as a science or if we are just going to run over the same old ground.