Happy New Year!

 

After a bit of a break, it is time to jump right in. The academic world can be a challenging space, depending on where one lives. Even if we take location out of the equation, there are plenty of political interest groups pressuring colleges and universities to crack down on whole degree programs, faculty speech, and student speech. I may not be a free speech absolutist, but I do generally err on the side of free expression, as the academy writ large tends to function better when faculty and students can speak their minds, with the caveat that there is an agreement on the fundamentals (what I call data or what everyone else would call facts or premises). So far, I've been pretty lucky, but I always know I am always one legislative session away from running out of luck. But I choose to be hopeful. I do know we will face some serious headwinds depending on what the next president of the US pursues in terms of student visa availability, the would Department of Education, and so on. In a sense, it is a good thing that policies governing curriculum are devolved to the states and in my sector often to college or university systems. Without some centralized means of enforcing faculty or students to follow a particular party line, the odds of pulling that feat off are very minimal, but not non-existent. If we're really fortunate, the incoming US president will prove to be about as lazy and ineffective as he was during his first term (2017-2021). 

This is an especially interesting time to be studying authoritarianism. I study authoritarianism from a psychological perspective as opposed to studying authoritarian political organizations and systems. I do think the work I and others who are far more well-known does inform how to understand how an authoritarian regime could become established and perpetuated, and that those in political science and history can inform us psychologists on how those systems might influence authoritarian attitudes among the citizenry. 

I have slowed down my research activities considerably these last few years. Initially, the pandemic threw a monkey wrench into my plans. Then it just came down to a series of personal family crises these last couple of years that have kept me out of the game. Thankfully, I am now essentially a senior academic, so perhaps it matters a bit less if I publish or not. I have been revisiting a couple topics recently and should be sharing a bit of that with you at some point later on in the year. 

In the meantime, happy new year!

Something I want to circle back to

 

 

 

There is a new three-part series on the Stanford Prison Experiment (SPE) on Hulu (Stanford Prison Experiment: Unlocking the Truth) that I will get to in a moment. The SPE was a huge part of the late Phil Zimbardo's legacy. As such, Zimbardo's experiment that was so influential for so long turns out to be far from trustworthy. The 3-part series is described in a recent Time article that is worth your attention. I would like to say more at some point in the future. Watch this space.

Useful Retraction Watch Article on Retracted Articles

I have discussed the problem of retracted articles continuing to receive citations before. In the process, I've mentioned some limited circumstances in which I can accept that a retracted article will continue to be cited. For the most part, my usual line of thinking is that unless the retracted article in question is being cited either to address some theoretical or methodological concern with a particular research area or is otherwise being used as a cautionary tale of what not to do (especially from an ethical standpoint), I see no point in a retracted article maintaining zombie status, and yet that occurs far too often, as many of pointed out before. 

Caitlin Bakker and Maria Zalm describe the problem and offer a summary of some solutions in their recent Retraction Watch article. As chair and member, respectively, of the NISO CREC Working Group their recommendations appear to be a step in the right direction. You can read their recommendations in more detail here. The bottom line is that retractions and expressions of concern need to be far more visible than they are currently.

And the urgency for making retractions and expressions of concern much more visible is quite salient today. Articles are almost always retracted for a reason: either their was some form of fraud involved or there was some noticeable incompetence that makes the results and conclusions one might draw from the work of questionable validity (to put it politely). When such work is cited without noting its retraction status, the citing authors are wittingly or unwittingly allowing a work that is no longer considered valid to maintain its air of false legitimacy. That can lead to us as professionals continuing to believe in and spread our equivalent of urban legends. And we have to recall that policy makers and attorneys use our work quite often, and not always for the public benefit. The recent Supreme Court case in which the plaintiffs challenged the FDA approval and safety of mifepristone for non-medical pregnancy termination using a retracted fraudulent study as the cornerstone of their argument. Although the Justices thankfully ruled on the side of those who want to keep mifepristone available, that outcome was not a foregone conclusion. We can think too of the damage done from a retracted paper that supposedly linked childhood vaccines to autism. We're decades removed from that retraction, and yet that discredited work is still the foundation for what has become a political movement that has set back progress on preventing dangerous communicable diseases. Although the stakes are usually considerably lower for many retracted articles, they all leave at least a few victims in their wake. Let's hope that these new guidelines, as well as the sort of databases created by Retraction Watch and others, make a difference. The fewer zombies we have in our respective sciences, the better.

Interesting podcast on Open Science and its enemies

Now that I have a couple of moments to breathe, I've been able to spend a bit of time on Bluesky (which is where a lot of academic Twitter landed after Elon Musk took over the platform and made it far worse), and reconnect with some folks whose work I respect. I have been gathering that there might be some trouble in paradise among the community advocating for Open Science. I've seen some chatter about a preprint that offers a very broad definition of what may be considered questionable research practices that have at least some in the community suggesting that the definition really is too broad. I may come back around to that, but if and when I have the time to do more than give the preprint an initial reading. Instead, I'll focus on a podcast that I had never heard of before entitled The Error Bar. It's a clever title, and the host, Nick Holmes, certainly strikes me as witty and open-minded. He is planning a three-part series on what he refers to as Open Science and its Enemies. This week's podcast is called the p-circlers. We can think of p-circling as reverse p-hacking according to Dr. Holmes. The upshot is that p-circlers hone in on a finding they do not like and then look for ways to make the finding seem suspect or so trivial as to not even be worth examining in the first place. Holmes focuses on one finding that seemed to create a stir, and a preprint that ends up resorting to p-circling behavior in order to explain away the findings as much ado about nothing. If you have 26 minutes or so to spare, it is a worthwhile podcast episode, and hopefully will provoke some thought. Since I have some proverbial skin in the game when it comes to presenting open science practices in my undergraduate methods courses, I want to make sure that our actions really do move our respective disciplines and sub-disciplines forward, rather than simply weaponize a series of recently developed tools for post-peer-review and in turn lead us to making the same mistakes as our predecessors. Anyway, give this episode a listen. I don't think you'll be disappointed.

P.S.: If you do not like listening to podcasts, Nick Holmes also blogs his podcasts. Here is the blog post for the episode on p-circlers.

Another moral panic debunked: The "woke" university

Judd Legum has a reasonably good explainer of the current moral panic that is targeting colleges and universities: the myth that these institutions function primarily for indoctrinating students into some sort of "woke" ideology. In a sense, this is simply a rehash of earlier moral panics targeting colleges and universities. In my day as an undergraduate student, there were still plenty of pundits who persisted to claim that universities were hotbeds of communist indoctrination - a notion I found quite amusing at the time. Later, I'd read that there was a noticeable downward trend in course offerings on Marxism and a downward trend in Marxist scholarship by the 1980s. My guess is that what really drove the moral panic of my day was student-led efforts to urge universities to divest from South Africa's Apartheid regime. Today's moral panic appears to have its origins primarily in DEI initiatives by universities and university systems starting late last decade, and more recently in student protests over the war in Gaza which is somewhat reminiscent of the anti-Apartheid protests from the 1980s and early 1990s. 

As Legum notes, many of the individuals who seem to be responsible for our current moral panic over "woke" ideology in universities happen to be billionaires (often in the tech sector) and political pundits on streaming services or in our legacy media. There are the usual anecdotes to make the reader's or viewer's blood boil. Anecdotes intended to provoke outrage aside, is there any data to back up the claims that our universities are indoctrinating our students? The answer turns out to be no.

Legum mentions an organization called Open Syllabus. The data collected by Open Syllabus can be quite helpful. Obviously the data in this case come from publicly available syllabi, which could be a limitation, but it at least gives us numbers to test the claim that our universities are too "woke" to effectively educate adult learners. It turns out that there are very few course syllabi that include terms like Critical Race Theory (or race more broadly), transgender (or gender more broadly) and so on. In other words, as Legum notes, it appears possible and even probable that students, even at elite universities, can go through a four year degree program without ever encountering any of the concepts that apparently cause our far-right billionaire class and punditry class to quake in their comfy slippers. We could certainly have a conversation as to whether a lack of exposure to structural racism, the concept of gender as a social construct, etc., is beneficial or detrimental to students who are preparing for careers in which they will likely work with and supervise a diverse set of individuals. There is certainly room for debate about the necessity and effectiveness of DEI initiatives in terms of fulfilling a university's primary mission and objectives. Clamping down on DEI programs, courses of study, and student speech with scant to near-non-existent evidence is far from conducive to an environment in which ideas are freely exchanged and challenged. 

The moral of the story is simple: if a claim is too outrageous to be true, it probably is. At that point, it is a really good idea to question the source or sources of the claim or claims, and do some digging to see if there is any trustworthy data to support their claim. If not, it is best to discount the claim as not valid and move on.

The problem of retracted articles continuing to receive citations

Now that I am mostly done with grading for the semester, it is time to turn my attention to a phenomenon that has bothered me for quite a long time, and will most likely continue to bother me. It is not uncommon for articles to get retracted for any of number of reasons. Sometimes, an error gets caught that invalidates the claim made by the authors, sometimes fraud is involved, and sometimes there is plagiarism involved. It happens. Life should go on. In an ideal world, once the article is retracted, that should be the end of its useful life. That article should receive no more citations. We don't live in an ideal world. I know. I am being Captain Obvious about that. The reality, as this article in Retraction Watch points out, is far more complicated, and concerning. Retracted articles may get cited less once the retraction is made public, but they still can rack up quite a number of citations.

I can think of a handful of reasons why an article might still be cited post-retraction:

1. In the months and year immediately after the retraction, papers by citing authors may have already been accepted for publication. It is quite likely that the citing authors would have no knowledge that a retraction was in the works. These are good-faith citations, and should be treated as such. The impact the retraction has on the papers by the citing authors may or may not have significant ramifications, depending on how much of their argument was anchored by the retracted article, or depending on if the effect size data from the retracted article was included in a meta-analysis. Usually the ramifications are fairly negligible, but we shouldn't always assume that to be the case.

2. The retracted article is cited as part of an argument for why the work of a lab, a principal investigator, and/or that PI's collaborators should be viewed with a very healthy dose of skepticism. Under such circumstances, we should expect that the citing authors will explicitly label the article or articles in question as retracted.

3. The retracted article is cited as part of a debate about whether or not a specific theoretical model is still viable in the face of one or more of a theorist's articles being retracted. Although I am not certain I would want to build much of a case that a theory is debunked because the theorist was either negligent or fraudulent in at least one instance, I can appreciate how such examples can provide some context. Under such circumstances we should expect that the citing authors will explicitly label the article or articles in question as retracted.

4. The retracted article is cited as a cautionary tale of what not to do. Retracted articles can be rich case studies in their own right, and guide scientists to steer clear of the sorts of mistakes or misdeeds that can lead to a retraction. No two retractions are exactly alike, although there are some overlapping patterns. My favorite retraction narratives are ones where the authors of a botched article actively work to get the offending article retracted. That said, when citing a retracted article as a cautionary tale, the article in question will be explicitly labeled as retracted.

Beyond those examples, I can't think of a defensible reason to continue citing an article that is for all intents and purposes removed from the public record (aside from the retraction notice specifying the reason for the retraction). I suppose it is possible that a citing author has cited the retracted article before it was retracted, and continues to cite it in their own future papers out of force of habit. That is not a good look. I see that sort of behavior as a sign that authors are not keeping current with the literature in their own areas of expertise. I can also imagine situations where the citing authors cite a retracted article in bad faith in order to further a theoretical or political agenda. That is also not a good look, and the societal ramifications are very, very concerning when citing authors simply hope that their readers don't bother to notice that they are basing their argument on findings that have been retracted for good reason. 

There is a simple solution for those of us who want to avoid citing retracted articles. The Google Chrome browser has a PubPeer extension that is really good at flagging retracted articles. Those who install and use that extension can be alerted to retracted articles well ahead of time, and that will give them ample opportunity to learn why the retraction happened and opportunity to find another article to cite instead. Be smart and use some simple tools that are freely available. In the process, you will help make your particular science more trustworthy. That's a good thing.

Postscript to the Preceding: Weapons Priming Effect and Potential Allegiance Effect

As I noted in my prior post, almost all of the experiments explicitly testing for a weapons priming effect (short term exposure to weapons increasing relative accessibility of aggressive cognition) involve the primary General Aggression Model theorists (Anderson and more recently Bushman) and/or their graduate students or associates. We are a very small group of individuals. The methods we use are strikingly similar, both in terms of independent variables and dependent variables. Over the years, we used very similar protocols when running our experiments. We used the same theoretical basis for our work. I can find one researcher citing any of our work independent of our clique who successfully replicated our findings (Korb, 2016), and she never published her Master's Thesis, as far as I am aware. With very few exceptions (e.g., Deuser, 1994), our experiments consistently found statistical significance. I often wonder if there were more independent efforts to replicate our basic findings, and if they were unsuccessful I would be curious to know what they thought happened, especially if they used protocols similar to the ones that we used. Otherwise, what we have is something of a niche area of inquiry that likely started and ended with just our cohort. I don't find that especially comforting.

Footnote: I am quite aware that Qian Zhang, who had a weapons priming effect paper retracted (Zhang et al, 2016), does still look at the weapons priming effect, and although his lab's findings on the surface are consistent with our own, I simply discard that work as I do not trust his reported descriptive and inferential statistics. Let's just say that GRIM and SPRITE tests tend to uncover mathematically impossible descriptive statistics in too many of his lab's findings. I shall leave it at that.