If you've read this blog long enough, you're familiar with the work of Qian Zhang of Southwest University in China. You are already well aware that there are some serious problems with many of the papers he has co-authored (either as a first author or a more secondary co-author) over the years. His more recent papers have been on the surface of higher quality, but it sometimes doesn't take much to realize that there are still substantial problems. Bottom line is that if you see his name mentioned here, it's not good news.
Case in point: Dr. Zhang has a new paper out that purports to examine the link between viewing prosocial cartoons and a reduction in aggressive cognition and behavior. I was alerted to this paper by Joe Hilgard. On the surface, a simple Statcheck run looked good. Initially I lamented the lack of tangible data to reproduce the analyses. Dr. Hilgard pointed me to where the data were stored (which kudos to this lab for doing so). I ported the dataset into my current version of jamovi and successfully reproduced the analyses reported in the paper. So far so good. Then I had that sinking realization something was still wrong. The data set only contained data for reaction time data for weapon images (which the authors use for the DV in the paper and analyses - a fact Dr. Hilgard had already arrived at before I did my work here). However, the authors should also had data on reaction times for neutral images that were not included in the data set. The appropriate DV would have been a difference score between reaction times for aggressive images (in this case, weapons) and reaction times neutral images. That difference score would be the proper measure of accessibility of aggressive cognitions.
As of this writing, the last author on the paper had been contacted, and I trust this last author to do the right thing here. At bare minimum, a reanalysis needs to be conducted in order to ascertain that prosocial cartoons really did lead to a decrease in the relative accessibility of aggressive cognition. As of now, the paper cannot adequately address that claim. There is this funny gray area between what we consider published and in press. The paper has already been accepted, and some version of it has been made available online. My hope is that the last author, an American researcher with a solid reputation, is able to get the matter resolved satisfactorily, however that turns out. Maybe a simple correction suffices. It is possible that a properly calculated cognitive DV yields the same basic findings as the incorrect cognitive DV. If it does not, then many of the conclusions of the paper may need to be rethought and rewritten. If so, the topic is of enough theoretical and practical interest that perhaps a sympathetic editor and publisher will still be okay with a corrigendum, regardless of how the ultimate findings flush out. If a retraction is necessary at this stage, it would be far less painful than after it is already officially in print. This is a matter of making sure that those of us who might still be tempted to conduct meta-analyses in this broad area of media violence have the correct findings when estimating effect sizes, that those who might be using this literature to advocate for policy changes have the right information before coming across as grossly uninformed. For the good of the order, I hope this matter is taken care of quickly. In the meantime, I'd warn against citing this particular paper unless and until at least some sort of correction has been published.
No comments:
Post a Comment