So a few weeks ago I noted that there had been four recent corrections to papers published out of the Zhang lab. It's time to turn to a paper with a fun history to it:
Tian, J. , Zhang, Q. , Cao, J. and Rodkin, P. (2016). The Short-Term Effect of Online Violent Stimuli on Aggression. Open Journal of Medical Psychology, 5, 35-42. doi: 10.4236/ojmp.2016.52005
What really caught initially was that there was a 3-way interaction reported as nonsignificant in this particular article that was identical to the analysis of a similar 3-way interaction in another article published by Zhang, Tian, Cao, Zhang, and Rodkin (2016). Same numbers, same failure to report degrees of freedom, and the same decision error in each paper. Quite the coincidence, as I noticed before. Eventually, Zhang et al. (2016) did manage to change the numbers on several analyses on the paper published in Personality and Individual Differences. See the corrigendum for yourself. Even the 3-way interaction got "corrected" so that it no longer appeared significant. We even get - gasp - degrees of freedom! Not so with Tian et al. (2016) in OJMP. I guess that is the hill these authors will choose to die on? Alrighty then.
So if you want to really see what gets changed from the original Tian et al. (2016) paper, read here. Compared to the original, it appears that the decision errors go away - except of course for that pesky three-way ANOVA, which I guess the authors simply chose not to address. Gone to is any reference to computing MANCOVAs, which is what I would minimally expect, given that there was no evidence that such analyses were ever done - no mention of a covariate, nor any mention of multiple dependent variables to be analyzed simultaneously. This is at least a bit better. The table of means at least on the surface seems to add up. The new Table 1 is a bit funky. I've noticed with another one of the papers that the Mean Square error based on the Mean Square information for the main effect and interaction effects that the authors were interested in would not give an estimate of Mean Square error that would support the SDs supplied in the descriptive stats. That appears to be the case with this correction as well, to the extent that one can make an educated guess about Mean Square error based on an incomplete summary table. Even with those disadvantages in papers by other authors in the past, I have generally managed to get a reasonably close estimate of MSE, and hence with some simple computations estimate the pooled standard deviation. That I am unable to do so satisfactorily here is troubling. When I ran into this difficulty with another one of the corrections from this lab, I consulted with a colleague who quickly made it clear that the likely correct pooled MSE would not support the descriptive statistics as reported. So at least I am reasonably certain here that I am not making a mistake.
I also find it odd that the authors now discuss that viewing violent stimuli had no change on aggressive personality - as if the Buss and Perry Aggressiveness Questionnaire, which measures a stable trait would ever be changed by short term exposure to a stimulus like a brief clip of a violent film. What the authors might have been trying to state is that there was no interaction of scores on the Buss and Perry AQ and movie violence. That is only a guess in my part.
These corrections, as they are billed, strike me as very rushed, and potentially as mistake-ridden as the original articles. This is the second correction out of this new batch that I have had time to review and it is as problematic as the first. Reader beware.
No comments:
Post a Comment