This is probably as good a time as any to mention that Zhang et al. (2016) is not the only experiment showing no apparent priming effect of weapons on aggressive thoughts. Arguably, anyone who follows this particular literature closely - and admittedly that is only a small handful of personality and social psychology researchers - is well aware of the two experiments in William Deuser's 1994 dissertation. To my knowledge, both experiments were well-designed and executed. Mendoza's (1972) dissertation also deserves mention. In that experiment, children were exposed to toys that had weapons or neutral toys. The cognitive dependent variable was content from participants' responses to a projective test. Those findings were non-significant for boys and girls across multiple sessions.
Those are the null findings that are at least in the public record. When we get to non-published results that found no link between weapon primes and aggressive thoughts (however manipulated or measured), there is so much that is unknown. What little I do know is probably more a matter of personal communication and hearsay. Unfortunately, those do not exactly lend to effect sizes that can be computed and integrated. For example, I am aware of one effort at the end of the 1990s to run an experiment similar to the sort that Bartholow and I had run, except that weapons vs neutral objects were subliminally primed. That experiment was a nonreplication. Given what we know now of the subliminal priming literature (which is littered with non-replications) this is not surprising. How I would love to get a hold of those data and protocols, to the extent that those might have been written up. I am aware of another weapons priming experiment from this decade that was designed to have a bit more ecological validity. As I understand it, the undergraduate student attached to that particular project bailed part-way through data collection, and the project was dropped. No way of knowing whether a replication happened or not there. There are quite likely other non-replications and half-completed projects stored on hard drives somewhere that no one knows about. From the perspective of a meta-analyst, this is especially frustrating as I am left hoping that publication bias assessments are adequately reflecting reality.
The cornerstone of the weapons effect narrative is that weapons reliably prime aggressive thoughts, which presumably leads to a chain of psychological events leading up to an increase in aggressive behavioral outcomes. What would happen if we needed to remove that cornerstone? My guess is that what remains of an already shaky narrative would crumble. As the state of the literature currently stands, the question of weapons priming aggressive behavioral outcome is inconclusive at best. That may or may not change in the near future.
So, what can we do? I am at least going to try to move the conversation a little. One way you can help me do so is if you have collected data where you have a weapon prime (weapon vs neutral images or words) and some cognitive outcome variable (reaction times on pronunciation task, lexical decision task, Stroop task, etc.; scores from a word completion task), talk to me. Better yet, make your protocols, data, and analyses available. If it turns out that aggressive cognitive outcomes are reliably predicted by weapon primes, that's fantastic. But if they are not, I think the scientific community and the public have a right to know. Seems fair enough, right? So if you have something that needs to be brought to light, talk to me. I'm easy to find on Twitter (I answer DMs regularly) or email (that's public record). I even have links to all my social media. Contact me at any time. I will get back to you.