Sunday, April 21, 2019

Maybe replication is not always a good thing

Check out these two photos. Notice the similarity?


The top image is from here. Since you'll probably be directed to the corrected version of the article, I will recommend going here to view the original, and also taking a moment to read through the Corrigendum. The Corrigendum is hardly ideal in this case, but seems to clear at least some of the wreckage. Moving on...

The second article comes from here.

So each article reports the findings suggesting a three-way interaction is not significant. In each case, the authors are wrong. I noted that with the weapon-priming article earlier.

But wait. There's more.

Notice that although each article is testing a different prime stimulus (the passage from the top image is one where weapons were primes, and the passage from the bottom is one where level of video game violence is the prime), samples representing different populations (youth in the passage for the weapons priming article and college students in the article examining violent video games as primes), and samples that differed at least somewhat in size, these authors miraculously obtain the same test statistic. A miracle, you say? Bull, I say. This is a case where it would be very helpful for the public to have access to data from each study, as there is reason to wonder how the same finding was obtained in each with the aforementioned differences duly noted.

There is something seriously rotten in the state of aggression research, dear readers. It is past time that we took notice, as there is a pervasive pattern of problems with published articles generated from this particular lab. If it were just one bum article, I could probably write it off as "mistakes were made" and let go. We're way beyond that. The real worry is that authors from this lab are getting in to collaborative relationships with other researchers in the US and EU who are generally reputable. I have to wonder how much their partners know of the problems that exist with their already existing published record. In some cases I wonder how their partners got chosen. The late Philip Rodkin was a researcher in bullying. He did not appear to have much of a presence in media violence research prior to teaming up with the Zhang lab. What expertise do more recent coauthors have with media violence research? How well do they know their new collaborators' work?

At the end of the day, I think it is safe to say that this is a case of unwanted replication. It is again a stark reminder that peer review is a porous filter. 

One other thing to note. Rodkin passed away in May 2014. The weapons priming paper was first submitted in July of that year. The violent video game study on which Rodkin appears as a coauthor did not get submitted until October 2015. I have no way of knowing what Rodkin's role was on either manuscript, and although I probably will speculate in personal conversations, I won't do so here publicly as that is probably irresponsible. There are certainly ethical ways to handle a situation where a contributor to a research endeavor dies, and I hope that the editors in each instance were made aware of the circumstances at the time of submission.

References:

Tian, J. , Zhang, Q. , Cao, J. and Rodkin, P. (2016). The Short-Term Effect of Online Violent Stimuli on Aggression. Open Journal of Medical Psychology, 5, 35-42. doi: 10.4236/ojmp.2016.52005

Zhang, Q, Tian, J., Cao, J., Zhang, D., & Rodkin, P. (2016). Exposure to weapon pictures and subsequent aggression in adolescence. Personality and Individual Differences, 90, 113-118. doi: 10.1016/j.paid.2015.09.017. 

No comments:

Post a Comment