Since the title addresses a very real question for me, it's worth asking, as a soon-to-be-retracted article was included in our weapons effect database. Fortunately, Fanelli, Wong, and Moher (2021) address this very question: what impact do retracted studies have on the conclusions we can draw from our meta-analytic findings? In other words, what are the epistemic costs? The good news is not much. The authors took a sample of 50 or so meta-analyses that had included at least one retracted study. The positives are really positive - findings tend to remain robust even after a retracted study or studies are removed. This is especially important to the extent that this finding holds when the retraction was due to something suspect in the methodology or in the data analyses, and not some other issue such as plagiarism. One thing that the authors do note as that much of the problem of retracted studies appearing in meta-analyses is preventable. Many of of the meta-analyses Fanelli et al. (2021) included in their meta-meta-analysis had included studies that had been retracted well before the meta-analyses in question were published. They have their ideas of some systemic corrections that would help. I would recommend including the PubPeer web browser plug-in as one means of screening for potential retractions early on in the meta-analytic database search process. It won't catch everything, especially to the extent that it is underutilized by psychologists, but it could help a bit. I would also recommend searching through the Retraction Watch database. Those are individual actions we can take, and take now.
Hat tip to Retraction Watch.
No comments:
Post a Comment