Is Most Published Research Really False?

That is the title of a recent chapter published in the Annual Review of Statistics and its Applications by Leek and Jager (2017). I am just now digesting this particular chapter, but at first glance, it appears that the talk about a reproducibility or replicability crisis in various scientific fields is a bit overblown. Why does this matter? We as scientists and consumers of science need reasonable assurance that the work produced by our fellow researchers is sound methodologically. If it is not, then we are in serious trouble. Leek and Jager hardly offer a rosy view of the state of research in a variety of fields, including mine. Their coverage of the OSF-sponsored replication attempt of 100 psychology articles published in 2008 (which sparked a good deal of consternation a couple years ago when it was published) discusses not only its contentions, but the weaknesses inherent in its attempts to replicate the studies its teams of researchers took on. The impression I get is that there is no real crisis, but we do need to step up our game a bit and make certain that the methods we use are appropriate, and that we are transparent in providing descriptions of our work (for replication) as well as data and code (for reproducibility). Not only do we need to communicate more clearly with the public (as I noted earlier) but we need to communicate more clearly with each other, and make certain that any statistical methodology we use is used wisely.

Preventing our slide into authoritarianism

The US is not an authoritarian state - yet. Are we at risk? Arguably so. Amy Siskind has been keeping track of subtle changes in the US since the election. Her latest Facebook post is here. Since the devolution of a state into authoritarianism is likely to be subtle, we may not realize we have crossed the proverbial Rubicon until it is too late. As scientists and scholars, we also have an obligation to be vigilant, as our ability to conduct our work with any degree of validity requires a free and open society that allows for a free and open exchange of data and ideas. It is also worth noting that there is a subset of American scientists who have signed loyalty oaths to the US Constitution, typically as a prerequisite for obtaining employment in various state and Federal agencies. I am among those who has signed such an oath, and I take my allegiance to the US Constitution and all it stands for quite seriously. Even if one has never signed a loyalty oath, I would advise making clear one's opposition to any noticeable apparent threats to our Constitutional order in the months and years to come. This is something any of us can and should do, regardless of our diverse and sometimes divergent ideological or partisan preferences.

Academics need to communicate more with the public

This article was on my Facebook feed earlier today, and I thought I would share it with you: Academics can change the world - but only if the stop talking only to their peers. The article is balanced enough to highlight how our collective knowledge base fails to reach the general public, yet takes into consideration how the academic world itself makes it difficult to take the time to communicate with the public. As an individual, I can certainly attest to the difficulty in balancing my workload and sharing new findings with non-academics. There are no real structural incentives to do so. That needs to change.

In the meantime, I will continue to use my blog to communicate my work as I can. I do have a few ground rules that I follow. One is that I will avoid dropping a bunch of statistics on you. Two, I am hesitant to say much about work that is on-going or is currently under review. I think I can provide broad summaries without too much difficulty. The main reason I say that is simply because reviewers and editors can and do get offended if they think they are reading work previously published elsewhere. So I may say in ordinary language that I ran an experiment that tested some phenomenon, and here is what I found and why it might matter. I am not publishing the details of the methodology, which frankly bores most readers, and am not publishing the data analyses. So I should stay in the good graces of those who review my work and the work of my coauthors.

Note too that what I share is truly on my own time. Again, just the nature of the beast. That means posts are often going to be irregular. I do have some fresh work on the weapons effect that I will share with you all in the near future.

Marijuana and Aggression: What Does Research Tell Us?

Current Attorney General Jefferson Sessions recently stated that there was a connection between marijuana and violence. That is his rationale apparently for making sure that marijuana remains illegal, even as more and more states legalize not only marijuana for medical use but recreational use as well. The argument might be plausible if there were an actual increase violent crime rates in states where marijuana had been legalized. However, the crime rate data suggest a different interpretation:

Denver saw a 2.2 percent drop in violent crime rates in the year after the first legal recreational cannabis sales in Colorado. Overall property crime dropped by 8.9 percent in the same period there, according to figures from the Drug Policy Alliance. In Washington, violent crime rates dropped by 10 percent from 2011 to 2014. Voters legalized recreational marijuana there in 2012.

Medical marijuana laws, which have a longer track record for academics than recreational pot legalization, are also associated with stable or falling violent crime rates.

Any violence associated with this particular drug is apparently connected with factors such as turf wars and deals gone bad that are common with the activity of trafficking the drug illegally. So in this case it isn't the drug itself causing violence, but rather the consequences of the illegal drug trade. This is something many of us who study aggression and violence from various disciplines would have known for ages.

Nor does experimental research suggest a link between marijuana intoxication and aggressive behavior in general. Note that in lab research, we don't measure violence, but we can measure physical aggression by having participants believe that they are delivering electric shock to another person. The higher the shock level, the more aggressively the participants behave. Myerscough and Taylor (1985) found evidence that participants who were given higher doses of TCH (the active ingredient in marijuana) were generally non-aggressive across provocation levels, suggesting that marijuana did not facilitate aggressive behavior. That followed up an earlier experiment in Taylor's lab suggesting that although alcohol intoxication facilitated aggression when participants were provoked, THC did not facilitate aggression under conditions of provocation.

More recently, Perna et al. (2016) showed that particpants who reported being either heavy alcohol or marijuana users showed an increase in aggression following alcohol intoxication and a decrease in aggression following marijuana intoxication. In addition, participants' subjective aggression was measured, showing that alcohol increased participants subjective aggression but that increase was not found among those in the marijuana condition. Perna et al. (2016) also provide a useful summary of the research literature, suggesting that much of the literature on marijuana and aggression (to the extent it exists) is largely hampered by very small samples, failure to include placebo conditions, and so on. The samples in the Perna et al (2016) experiment are also a bit smaller than I would want to see. However, as a generally well-designed exploratory experiment, it does suggest that we think twice before accepting the pronouncements of a politician on faith. The findings are also generally consistent with much of the earlier experimental and cross-sectional research that I am aware of.