Monday, July 28, 2025

Postscript to the preceding

This deserves a proper quote, regarding the decline of US academia:

Some of this was foreseeable. The high school graduate demographic is shrinking, and college and university tuition rates were growing unsustainably. But Trump’s personal attacks on Harvard and other elite universities are erasing the old social contract between the government and academia. On top of that, Trump’s attack on immigration is discouraging foreign student enrollment, and these students were paying full tuition, subsidizing tuition for domestic students. The US higher education brand is tarnished by Trump policies and may never recover. US talent—graduate students, postdocs, faculty—are looking abroad and will take their intellectual capital with them.
Not much can be done about the decreased number of available high school grads (still our major source of students at most universities and colleges). The same thing occurred after the Baby Boom generation reached their college years. Give it a generation and there will be another increase in available potential traditional students, assuming there are sufficient colleges and universities left in the US by then. The unsustainable tuition rates are definitely an own-goal by our politicians starting approximately during the Reagan era onward. Don't even get me started on the student loan bubble. Trump does manage to make things worse, and I say that knowing that some of my readership will be offended. So it goes. The reputational damage is something that I am not sure can be undone. I am usually optimistic, but I don't really see how US higher ed comes back from what will be a catastrophic four years. Once universities begin obeying in advance, allowing a White House occupant (of any party, but for now since it's Trump we'll keep our focus there) to extort them, they simply become institutions that are not to be trusted. Fear and expedience are not acceptable excuses, and those administering these once-fine institutions will not be remembered fondly, to put it politely.  

The end of the golden age of higher ed...some thoughts

Joel Eissenberg, who is one of several writers for the Angry Bear blog, posted today on the end of the golden age of academia. The trigger for the post was undoubtedly an article on academic layoffs at colleges and universities that appeared in the Boston Globe (Joel links to that article if you are interested - I usually don't bother with the Boston Globe due to paywall, and my own paycheck can only sustain a small handful of subscriptions). 

Joel is a professor emeritus, and his experiences are worth reading about. To an extent, what he experienced as a student tracks with my own. Going to Cal State Fullerton was dirt cheap when I was an undergraduate. In-state students paid some nominal fees, but no tuition, although those fees got steeper around the time I graduated with my BA. Funding for public colleges and universities began to dry up as the 1980s progressed. In California, part of the problem was the passage of a series of initiatives that limited the state's ability to collect revenue, which meant that public education took a hit. You could say that by the time I became a college student, the golden age of academia in the US was already in the rear view mirror. Many of us would not have noticed at the time as facilities were still being maintained, a wide variety of courses and degrees were still being offered - including in the traditional liberal arts that are often derided here in the US. If the student government wanted to fund a Henry Rollins band concert, it could without a problem. Heck, Cal State Fullerton had Olympic-quality men's and women's fencing teams (I was friends with a few members of the women's team). 

Austerity took its toll, and into the 1990s (I stuck around to get my MA in Experimental Psychology) I noticed a decline in maintenance and upkeep (overflowing trash cans became the norm for my last couple years) and a decline in cultural, sports, and academic offerings began. My own funding for my MA degree came from a grant awarded to Ron Riggio, Tom Mayes, and Carolyn Kubiak to study the role of internship experience on managerial skills (individual and group decision-making, interview skills, etc.) above and beyond classroom education. At the time the grant began, there was an understanding that the PIs could apply for an extension beyond the three years they were awarded. That made sense as many of the students that made up Cal State Fullerton's population were non-traditional students who may only have been going to school part-time. In the middle of my second and last year of my MA, we got the bad news that the pot of federal money that made that particular grant possible was going away and that we would have to wrap up sooner than the PIs had hoped. If nothing else, that did motivate me to work around the clock to get my thesis successfully defended. I had good reason to believe there might not be enough money to fund my position much beyond that second year. I managed to get full funding at Mizzou for my PhD, but as we ended the 1990s, the vibe I was getting was that the days of funding students beyond year five were over. Thankfully, I was done with my dissertation before I could find out what no funding looked like. Increasingly, it became difficult to fund doctoral students unless there was grant money to fund them. The US government was no longer interested in funding the land grant institutions and the states weren't necessarily motivated either. We've seen what that means for tuition (my youngest daughter is living that dream right now). But those who advised me were optimistic about higher ed's prospects as the new century approached: surely a boom in student enrollment (which did happen) would lead to an increase in state and federal support. Yeah, that never really happened. 

I've spent my entire career as a full-time professor witnessing the decline of the university as a viable place to study and work. Even the small universities where I've made my living once upon a time offered a full range of degree offerings and handled all custodial and maintenance of the physical plant in-house. Budget cuts led to outsourcing, and layoffs of those staff members, even as student enrollment increased or at least remained steady. I get the feeling that at least on the surface the flagship public universities and elite private universities were able to maintain some semblance of "the golden age" as long as the grant money flowed enough and as long as there were plenty of international students who viewed our graduate and professional degree programs as the gold standard. I'd say those days are truly over. And in reality, the financial pressure for grad students in an era where full funding could not be assured as tuition continued to increase to a greater degree than the inflation rate was taking a toll. Grant funding was getting harder to come by, especially as the Tea Party era took hold in the 2010s. And then came Trump. Now the rule is that funding is only given contingent on being sufficiently "non-woke" (whatever that might mean), and international students are no longer welcome in the eyes of the current federal government. Losing international students will end not only individual research programs, but likely whole degree offerings and the careers those degree offerings sustained. 

I am not optimistic about the prospects for higher ed going forward. Even if the US survives the next few years, the damage to the reputation of our education system will be done. We are already seeing the beginning of a brain drain in the US, and that will likely continue. As someone in the sciences (social sciences specifically), I am still optimistic about how the sciences will progress. The US government can end a lot of the scientific work that universities once did without interference, but there are universities in other nations that will no doubt pick up the slack. It just means that aspiring scientists in the US will want to do their graduate training outside the US and will want to look for career opportunities outside the US. And depending on one's ideological perspective, the response might be "who cares?" By relinquishing its role as a leader in scientific research, US citizens will no longer get to benefit directly from new advances. Their counterparts in China, India, and the EU (which seem well-positioned to hire the very scholars the US no longer wants) are already swooping in to hire scholars who would have otherwise remained in the US. Americans will get used to no longer being first in line. They have no choice. 

Thursday, July 17, 2025

A few musings

At some point, I need to re-read Allan Bloom's The Closing of the American Mind. It was a timely critique of higher education when it was published back in 1987 (that book was a surprise Christmas gift from my parents back in the day), and perhaps is more urgent today. Actually, there are a lot of books that would fall under the "cultural criticism" category that deserve a re-read as time permits.

 I won't try to summarize Allan Bloom's book for this post, as I would be drawing on a nearly 40 year old memory, and that's not something I wish to endeavor, but I can discuss a few things I think need to be reformed if higher education is still going to meet our needs as a society. 

Let's start with one elephant in the room: the K-12 system in the US has been in decline for the duration of my adulthood. Think about this: none of my grandparents ever graduated from high school. They either worked small family farms or did manual labor. And yet, they had a better grasp of the arts, philosophy, and literature than the typical college graduate of today. What happened? Too much to mention in a short blog post. But I have noticed a general trend away from teaching critical thinking and more and more teaching to the test. The George W. Bush era arguably accelerated the descent of K-12 education. Note that I am not saying that our instructors are sub-par or about to make some crackpot claim about schools being too "woke" or any other moral panic verbiage. Our nation simply does not offer the educational opportunities that even my age cohort would have taken for granted. By the time one graduates high school, one likely has never been to an art gallery on field trip, or to a local symphony (assuming one still exists), or to a play put on by a local community theatre company. Maybe the kids get exposed to some Shakespeare, but that's hardly a given. If asked to evaluate the merits of an argument, my impression is today's high school graduate is probably going to parrot talking points from a podcaster, peers, or perhaps extended family rather than independently look at the claims made on each side of the argument and offer an informed evaluation. The sort of reflective thinking that these nascent young adults are perfectly capable of demonstrating has been left to go to seed. 

I used to say that my job was undoing the damage caused by the K-12 system, as that system is overwhelmed with all sorts of metrics and standardized tests - none of which have much of anything to do with the sort of reflective thinking we once expected beginning college students to have. Of course, higher education is also in decline in the US, and is probably going to get worse at least for the short and medium term. I am part of that last generation who went to college not just to learn the ropes before finding that entry level managerial gig, but to actually get a well-rounded education (and yes, of course to party as well - social skills are fundamental). The very degree programs that made all that possible are increasingly on the chopping block. I will accept the claim that getting a philosophy degree is not exactly "practical" in terms of getting that first job after graduation, but the skills learned are crucial on their own merits, and have the fringe benefit of making one a better employee to the extent that a job requires critical thinking skills. Majoring in a language may not seem like the obvious strategy for a job where you will work in a cubicle, but one learns in the process not only a different way of thinking but also a knowledge of a culture or set of cultures one would have been ignorant of otherwise. Faculty in higher education aren't exactly reinforced for educating their charges in critical thinking. We're busier than ever completing paperwork to justify our own existences based on some metrics devised by some political appointees that are tangential to education at best.

Institutions of higher education in the US have turned their backs increasingly on the expectation of excellence. Oh, don't get me wrong. There is tons of lip service given to professional development. Just don't count on support for your efforts if you end up on faculty. You'll be reinforced for looking "good enough" based on whatever metrics are used as evaluation and nothing else. The system reinforces mediocrity, especially for those of us who make our lives nurturing our undergraduate students. I have certainly learned the hard way that the sort of ambition that leads to advances (however minor) in our respective fields is unwelcome. Ambition to become an administrator is possibly more accepted. Ambition that leads to errors and fraud are also apparently okay as long as nobody looks to closely. That's just the way it is anymore, as I see it.

We didn't get to this already low point overnight, and I am not sure I am the best person to talk to regarding how to improve the educational system so that it at least continues to function in the service of further developing our global civilization. Education needs to be valued. It's a fundamental need. That means as a society we need to do something radical: fund education from preschool all the way to colleges and universities. At all levels of education, age-appropriate experiences that foster critical thinking, that provide a basic understanding of how our society works (government, civil society, etc.), and foster a genuine appreciation of the arts and humanities as well as the sciences are a must. A student graduating from high school with some basic knowledge of Plato and Aristotle as well as Lao Tzu and Sun Tzu would be a good start. A student graduating high school who understands that the anime they were watching referenced a specific work of art or a play would be a good start. With more funding comes greater expectations, and we as a society need to be up to the challenge. But to get there, so that I can actually do my job the way I intended when I began teaching full time about a quarter of a century ago, we need a massive cultural shift toward valuing what educators do, encourage educators and students to express independent views (whether popular or not), and funding the institutions charged with that task. W do that and we might just have some hope of surviving as a society for the duration of at least this century. On our current path, we'll be lucky to have taught a generation of students how to craft AI prompts to generate reports that have no substance. I don't see how we survive as a society under that dystopian set of circumstances. I wish I could be more optimistic.

Wednesday, July 2, 2025

How to lie with statistics: ICE edition

When anyone makes a claim that assaults on ICE agents have increased roughly 400% to 500% above last year, that will raise some eyebrows. As framed, it's no wonder these individuals would feel the need to hide their identities and faces. But what if there were less to the story than meets the eye? Techdirt has the dirt (pun intended) on the government's claim. It turns out that for the first half of 2024, there were 10 alleged assaults on ICE agents. For the first half of this year, that number is 79 alleged assaults. Keep in mind that just because an assault is claimed does not mean it was actually an assault (sometimes the DOJ just makes stuff up - I swear these folks would have had productive careers as social psychologists). Even if we take the DOJ claims at face value, the raw number of alleged assaults is really minuscule when we factor in the thousands of encounters ICE agents within each of those time periods have with the public and we would want to factor in that there may have been more encounters this year so far. In other words, once we start factoring in the baseline data for assaults and the sheer amount of ICE agent activity in a given year or half year as is the case, this "dramatic" increase in alleged assaults looks more like statistical noise. This is also my friendly reminder that when public officials use numbers like these to outrage or frighten you, it is important to look under the proverbial hood, kick the tires a bit, and really dig into the data to determine if these officials are being truthful or if they are trying to manipulate you by lying. I'm pretty jaded regarding politicians and political appointees, so often my default is to not believe them until they show verifiable receipts. 

Friday, June 27, 2025

Reckless Disregard for the Truth

I've been following Andrew Gelman's blog for a while. It is typically worth my while. Although he refers to this particular post as a rant, as rants go, it's quite substantial. It's not a purely scientific post, but it is one in which Gelman makes a connection between various sectors of society (science/academia, medical, and law enforcement/political) when it comes to various actors engaged in a reckless disregard for the truth. In each case, Gelman outlines how the individuals involved stretched the truth to the breaking point, often with minimal repercussions. Misleading spin on research results? That's just "normal science", right? Testimony as an expert witness that makes you look idiotic or dishonest? The paycheck will more than cover the ding to one's reputation. ICE "agents" (there are much more accurate words, although they will sound derogatory) claim being "assaulted" by elected officials when every bit of recorded evidence screams the opposite? The White House is above the law, apparently, so that's also business as usual. As Gelman notes, "the scandal isn't that it's illegal, but rather that it is legal." 

So, that's the world we've been living in. Doesn't mean we have to accept it, right? Think of the repercussions to what we're going to call a reckless disregard for the truth. In my own line of work, one can publish findings that are obvious bovine fecal matter, but as long as it gets through peer review that's good enough, we're told. Here's the thing: some people will actually believe what is published and they will go off and use those dubious findings to guide their lives and policy decisions, leading them to take some wrong turns along the way. In other words, the outcomes are not necessarily no harm no foul. To the contrary, the outcomes could be more nefarious, as the actors in the scenarios Gelman laid out muddy the waters, blurring the bounds between fact and fiction, leading to zombie ideas spreading, harming the population at large in the long run. He's also right that we can't stop folks from recklessly disregarding the truth, but we don't have to stand by and tolerate it.  

Thursday, June 12, 2025

What are the odds this scholar remains in the US?

After four months in ICE detention facilities, a researcher who fled Russia, Kseniia Petrova, has been released. What did she allegedly do to deserve the revocation of her visa? She transported harmless samples as requested by her supervisor at a Harvard lab (pro tip: those of us who have ever worked for a principal investigator at a major university do not say no to requests to do something legitimately within the scope of the lab's research if we wish to keep working for that particular lab - trust me on that). Once her visa was revoked, she got to spend several months in US gulags. She decided, understandably to fight the deportation (as that would involve her ending up in another gulag in Russia for speaking out against the Russo-Ukraine war), and then got slapped with criminal smuggling charges. In other words, what was at most a minor paperwork blunder could potentially land her a couple decades in prison or worse. For the time being, she is not cleared to go back to work at her lab (unless she gets her visa reinstated), and she is still facing those smuggling charges, even though any sensible judge would likely throw that case out on its merits. Will she stay in the US long enough to face those charges? I seriously doubt it. She is already looking at options to continue working outside of the US. In other words, the US will lose a promising scientist because of a very hardline xenophobic approach to non-citizens working in the US. She is not the only international scholar to get placed in one of the US's ICE gulags. And that is a reminder that the US is no longer a place for promising scholars to start their careers. In the case of Kseniia Petrova, I doubt she'll be in the US any longer than it takes to pack up her belongings and take the next research job available. I don't blame her. 

What the US is losing

I just read this article on Pro Publica that I think every concerned US citizen and resident should read: Science Shattered. Although the focus of this article is on NIH grants and the politicized freezing of many of those grants, it is safe to say that something similar applies to other US grant funding agencies (e.g., NSF). To put it bluntly, the current White House "administration" (if you can call it that) has politicized the grant funding process, defining what is and what is not "scientific" based upon some ideological litmus test or perhaps sheer disdain for the work scientists do. As you read through this article, really spend some time digesting the stories these affected scientists tell. The termination of their grants obviously has negative implications for the careers of the principal investigators, their post-docs and graduate students, and any support staff involved, not to mention any third party vendors. But get past that for a moment and think of what those of us living in the US lose: basic research that could lead to medical breakthroughs or insights on human behavior that may have implications for therapeutic interventions. In this current regime, any research project with a word such as "gender" so much as mentioned is on the chopping block for being too "woke". In other words, if your research does not follow the current ruling party line, it has no place here.

Once those grants are gone, the luckier labs will find an alternative source of funding and continue to do their work in the US. Others will find their research programs snuffed out for good, thus ending careers of scientists at all levels from those who are principal investigators all the way to post-docs and graduate students depending on that funding for their own related work, including theses and dissertations. Many affected scientists will simply leave the US and continue their work elsewhere. The implications for the public are dire. The work affected discussed in the Pro Publica article will impact public health, and not for the better. Work that could have led to more equitable health outcomes, once abandoned, will ultimately lead to a loss of lives that was entirely avoidable. Then again the ruling party not only does not prioritize such research but has shown a doctrinaire opposition to the work being carried out at all.

The reality is that we have had federal grant funding through NIH, NSF, etc. for as long as I can recall. These agencies have objective criteria for awarding grants that are divorced from partisan politics. These agencies fund projects where the principal investigators have demonstrated that their work is going to be rigorous, doable based on the funding request, and will make a significant contribution to that specific scientific discipline. That's not to say that politics is not involved, as whoever occupies the White House and the composition of Congress certainly determines how much grant funding will be prioritized. Some years there is more funding available. Some years there is less. That said, under both Democratic and Republican administrations, at least until the Trump era, there has not been a political litmus test applied to projects in order to be funded. Some Congressional representatives have voiced skepticism about the value of specific funded research projects, but the general zeitgeist has traditionally been one of not dictating what can and cannot be funded - only the amount of funds to allocate for grant funding. My understanding of science that has to toe a particular ruling party line tends to be disastrous (see Trofim Lysenko and what came to be known as Lysenkoism during the Stalin era as a cautionary tale). The political appointee in charge of the Department of Health is well on the way of creating his own version of Lysenkoism. So we will lose out on new and more effective vaccines, better treatments or cures for any number of diseases, effective means of prevention when it comes to health and mental health problems, etc. 

For a long time I have told my students that mixing partisan politics with science is an awful idea, and that eliminating funding for research that might not be convenient for partisan reasons is nothing short of stupidity. The bad news in the US is that those of us who have to live here are going to lose access to research funding that we once could count on, and that ultimately it is the citizens and residents of the US who will suffer as a result. The good news is that the sciences, like nature itself, abhor a vacuum. If the US government has decided to embark on a self-imposed lobotomy (to use a phrase Carl Sagan once used to describe the waning days of the Roman Empire as it descended into a dark age), other nations will use the opportunity to fill the void. That means a lot of US scientists will end up moving to the EU, China, India, or wherever the funding exists for them to do their work. So in the long run, the sciences will be just fine. The bad news for any of us in the US is that we will find ourselves no longer first in line when a new discovery has tangible applications. Whoever discovers a cure for some form of cancer may well be located outside the US, and it is quite likely that those living in the US will never have a chance to benefit from that discovery, or if we do, only after a long wait. That will simply be the new normal.

Most Americans have been ignorant about what the sciences do for a very long time. Science education at the K-12 level has been a joke for decades. Science reporting has been dramatically scaled back since the 1990s. Americans live with the benefits of the discoveries made by scientists who had NIH funding (or NSF or equivalent federal agencies) without having any understanding of what was involved to make those discoveries or applications possible. Ignorance is not bliss. Instead, those who do not understand what they have been given and what they stand to lose are prime targets for demagogues, and we are experiencing the consequences as I write this. I have no doubt that some version of "science" will continue in the US, but it will become increasingly poorly funded and done in a way in which the answer is predetermined and the challenge is to create the "evidence". American science will become increasingly less trustworthy. I really want to be wrong here. I hope I am wrong. Regrettably, the only way to be wrong is if the US is able to successfully undo the damage that three and half more years under the current White House regime will do. I am not exactly optimistic that American politicians are up to that particular task. But hey, there's always the EU, China, India, UAE, etc. For scientists, these are all viable options. The US will no longer be a leader in the sciences. Maybe it was inevitable. 

 

Wednesday, June 4, 2025

Who am I, anyway?

A while back, I had ChatGPT generate a brief biography about me. Admittedly the prompt was not especially sophisticated: "Who is Arlin James Benjamin Jr., PhD?" Here is what it came up with"

Arlin James Benjamin Jr., Ph.D., is an American psychologist and professor emeritus at Humboldt State University in California. He has made significant contributions to the field of psychology through his research on intelligence, creativity, and personality.

Dr. Benjamin is best known for his work on the HEXACO model of personality, which posits six dimensions of personality: Honesty-Humility, Emotionality, Extraversion, Agreeableness, Conscientiousness, and Openness to Experience. His research on this model has been cited extensively in the field of personality psychology.

In addition to his work on personality, Dr. Benjamin has also conducted research on intelligence and creativity, particularly in the area of divergent thinking. He is the co-author of the Torrance Tests of Creative Thinking, a widely used assessment tool for measuring creativity in children and adults.

Dr. Benjamin has published numerous articles in peer-reviewed journals and has received several awards for his contributions to the field of psychology, including the 2013 Distinguished Contributions to Creativity Research Award from the American Psychological Association.

 I had no idea that I was apparently even on faculty at Humboldt State University, nor that I had any of the above accomplishments. If you wonder why I remain skeptical about AI, here is just a taste. Using AI to generate a professional biography was an amusing exercise, but it clearly laid out a serious problem: what is generated is far from accurate (I know that is clearly a Captain Obvious statement). In the meantime, I continue to toil in relative obscurity, and not using AI in my writing.

 

Saturday, May 31, 2025

Speaking of zombies: when public officials use AI to write their papers for them

This is embarrassing. Yes, it appears that the Make America Healthy Again (MAHA) Commission report was riddled with errors, including some non-existent citations and misrepresentation of other cited studies in its narrative. My initial impulse when I first read about the errors was that the report was probably largely "written" with AI instead of by the authors of the report themselves. It turns out my gut feeling was right on the money. Washington Post (paywalled, unfortunately) has an article showing a fair amount of AI's fingerprints on the references cited. I can't say I am really surprised. These fools apparently expected that no one would be the wiser. Remember that old saying, "you can fool some of the people some of the time but not all of the people all of the time"? I'd reckon that saying is applicable here. 

If the individuals responsible for this MAHA Commission report were students in one of my classes and I noticed the apparent AI influence, I would have no choice but to assign a 0 to the paper citing plagiarism concerns - with examples clearly identified. Regrettably, these are not my students but rather are public officials who wield a great deal of power over our own health outcomes in the US, and it is highly probable that the base supporting the current White House administration won't care about that little matter of academic integrity. This is not the first time that a high-profile public individual has been caught using AI to do their work, and it is safe to say it won't be the last time. I am asking those in the scientific community and the general public to be vigilant and to call out this form of fraud when they see it. That is especially crucial when the documents generated by AI are going to be used to make decisions that affect all of our health outcomes.

Question: Can we rid science of retracted articles?

I saw this opinion article on the problem of zombie papers and some possible solutions (i.e., retracted papers that continue to get cited and have influence) thanks to Retraction Watch. You can read the original in French here. Marc Joets makes some useful points here. Yes, we definitely have a problem. Part of the problem is that it takes a long time on average to get a problematic article (i.e., one that has error-ridden data reporting, fraudulent data reporting, or plagiarism) retracted. Apparently from date of publication to date of retraction, you're probably expect three years before a problematic paper is retracted, give or take. Apparently there are regional differences in time taken to retract an article: American and western European based journals do so more rapidly than elsewhere. Subscription based journals tend to retract articles more rapidly once a problem is identified than open source journals. In other words we need to keep in mind that there may be variations in culture and editorial practices at play. 

I've mentioned retractions before, and zombie articles before. In our various scientific fields, zombies are a legitimately concerning problem. As long as they are cited, they risk infecting not only the specific scientific discipline in question but also public discourse and policy. If you are living in the US right now, you probably know that a fraudulent and retracted article that spread some outright lies about the safety of childhood vaccinations has led in a matter of a couple decades to mainstream an anti-vax movement that is now in control of our own federal public health agencies. In this case, the consequences are life and death as the government is no longer as interested in containing a deadly measles outbreak. In my corner of the scientific community, the stakes may be considerably lower, but zombie articles can still infect public discourse and policy in ways that are not in the public interest. 

So, what to do? The answers in this editorial are ones that strike me as common sense at this point. Making data and research protocols publicly available can help to catch mistakes and fraud early enough to nip the problem in the bud. Better plagiarism detection tools are mentioned as well. Ultimately the author notes that there is no one-size-fits-all solution. But in broad brushstrokes we can expect that efforts to beef up transparency help. Efforts to improve reproducibility - requiring pre-registration of research protocols and offering evidence of replicability -are also necessary. Any of these practices can help detect errors or problems in a more timely manner. In addition to better plagiarism detection tools, the author suggests that each journal have its own panel that can objectively handle instances of fraud or serious errors as they occur. Finally, the author argues for making the fact that these articles have been retracted more visible in order to minimize the impact of retracted work. That strikes me as a solid idea. I still get a sense that retractions are not nearly as visible as they could be. Those with the PubPeer browser extension might be a bit more wise to retractions, as are those who use the Retraction Watch site's own retraction database. But how many of us are actually using those resources currently and consistently? I wonder. 

I wish that instructions for eliminating zombies from our sciences were as simple as "destroy the brain or remove the head"* but alas they are not. I remain a cautious optimist however.

*The reference in that quote was specifically to "Shaun of the Dead" which is a personal favorite of mine, but probably would refer to most zombie films and series I've seen over the years. 

 

Saturday, May 10, 2025

A cyberbullying paper with some concerns

To preface this: studying cyberbullying and examining how bystanders respond to cyberbullying if they notice it is a worthwhile endeavor. I'd say the same about bullying research in general. That said, just because the endeavor is generally worthy, one needs to be on the lookout when reading original empirical research reports. 

Let's just dive right in, starting with a link to the paper, so you can read it for yourself. Next I will provide the link to the comments on PubPeer.

Now, this is a decade old article at this point. In social psychology, that makes it ancient history to some. But here's the thing: no matter how much time passes, this is the sort of thing that will end up included in relevant meta-analyses. If something goes amiss in the original report, it will have some influence on the effect size estimates and on estimates of publication bias. That's where my concern is focused. 

If you look at PubPeer, there are several comments on the article, dating back to August 2022. Admittedly, I don't utilize PubPeer as much as I would like, and sometimes I miss something concerning to me. That's my reminder that I should go ahead and start using the PubPeer browser extension again. Comments 1 and 3 are arguably the more substantive of the comments. The last two comments merely reinforce what the initial commenter observed.

What were those comments? Let's review, starting with the first:

I have a few questions about this well-cited paper.

1. The abstract states: "Most participants (68%) intervened indirectly after the incident and threat were removed", but I cannot find this statistic or the n frequencies to calculate this statistic anywhere in the results.


2. Table 1 has the heading "Noticed cyberbullying" and then the subheading "Yes" and "No", implying that the columns are to compare those that noticed the cyberbullying versus not. However, the ns listed correspond to how many participants directly intervened regardless of whether they noticed cyberbullying or not.


3. I inputted some of the values provided into t-test calculators, which resulted in slightly different statistics. Due to the lack of information on the analytic plan/tests used and information about missing data/data normality it is difficult to discern the causes of these discrepancies.


4. 8% of the sample was omitted from analyses for being "suspicious" but no other information is provided. It is difficult to assess the quality of the sample or replicate the paper without further clarifications on how so many participants were determined to be "suspicious".

And here are some screenshots with highlights in yellow to point out what appeared to be problematic:


 


 


 You can enlarge the images by clicking on them. I will admit that like the initial commenter, I found the article's prose to be a bit challenging to follow. Some of that can probably be chalked up to the many edits and re-edits that happen when going through the peer review process. So it goes. But, there are some apparent typos that can't be chalked up to the inevitable compromises that happen in an effort to satisfy a reviewer or an editor. 

The first criticism of the article appears in the abstract, where the authors claim that 68% indirectly intervened is a fair one. This statistic gets thrown in there, but figuring out where it came from was not exactly straightforward. Eventually, buried somewhere, I was able to suss out that of the 221 participants who were included in the analysis, 150 of them noticed that bullying behavior was happening, which yields 67.87% of the total sample (or 68% for short). However, keep in mind that the statement that follows is the same basic percentage used to determine indirect intervention, which would not necessarily been the same percentage. Those who did not notice the bullying behavior would by definition be only able to indirectly intervene, if that makes sense. So let's see what happens when we remove 71 individuals who did not notice any bullying. I am unable to find where that 68% figure comes from. If we have 127 participants who noticed the bullying but only indirectly intervened and 23 who directly intervened, the percentage of those only using indirect interventions is 84%. That percentage goes up tremendously if we use the whole sample. So there's that. 

Table 1 could have arguably been constructed better, and I can see how a casual or even careful reader of the article could find themselves doing a double take. Looking at the labels, I can see how a reader could easily be misled into thinking that a whopping 198 participants did not notice any bullying, which is not what the authors communicate elsewhere in the article. Not great. There is a Chi-Square analysis in that table that does not seem to appear elsewhere in the manuscript. We'll come back to Table 1 in a moment. Going back to the narrative relevant to Table 1, there is another Chi-Square mentioned, right after a beta-weight each with an identical p-value. That should be explained to the article's readers much better than it was. It strikes me as sufficiently obvious that folks are much more likely to directly intervene if they notice something occurring than if they don't, but where that 4.62 times greater figure comes from is not quite explained. I am guessing that the authors somewhere had seen the direct interventions where the participants noticed the bullying and the direct interventions where the participants didn't notice the bullying (I guess that could have happened given the way the study was set up), and then did some quick back of the napkin division to sort that out. We just don't get to know what those numbers were. Okay. I said that and now it is time to let go.

The initial PubPeer commenter noted having difficulty reproducing the t-test results with the information made available in the manuscript and table. That should not have happened. One problem with the t-tests tied to Table 1 is that they appear positive in the table but are reported as negative in the manuscript. That is a bit jarring. Add to that a problem another commenter found in the manuscript: two of the cell means reported in Table 1 do not match the corresponding cell means in the manuscript itself. That would make reproducing a t-test challenging. I ran a SPRITE analysis on the cell means and standard deviations, making the assumption that the indirect interventions were scored on either a 1-to-4 or 0-to-3 Likert Scale (depending on the question) and that each intervention was based on a single item (I saw no information to the contrary in the manuscript). I made some assumptions that all 221 participants were included in those analyses (150 who noticed the bullying vs. 71 who did not). When I used those inputs, two of the cells appeared to report standard deviations that were mathematically impossible. To the extent that a t-test calculator would need accurate mean and standard deviation information, that is troubling. I will insert the image highlighting the discrepant means in the manuscript and you can compare those to Table 1 (above):

 

A later commenter highlighted some specific problems with two F-test results based on a Statcheck scan. Statcheck can reproduce a p-value based on the test statistic and degrees of freedom with a reasonable degree of precision. The two F-tests tied to Table 3 have slightly different p-values than what Statcheck computed. That's not the end of the world. Typos happen, and in any event, one would still reject the null as the authors had done. What is odd is the discrepancy between the degrees of freedom reported in the manuscript versus what was reported in Table 3. 

I get the feeling that a lot of the apparent errors could be cleared up with access to the original data set, if it still exists, along with the analysis plan. Unfortunately, as the initial commenter on PubPeer noted, there did not appear to be any analysis plan reported. And it goes without saying that the article in question would have been published at a time when journal editors were still fairly laissez-faire about publicly archiving data. So I am having to trust that somewhere the original data used to compute the analyses exists and is accurate. That said, it's been a decade since publication and likely considerably longer since the study was originally conducted. It's been said by others far wiser than me that we psychologists have historically been poor stewards of our original data. If the original data and analysis plan haven't been deleted, I'd be pleasantly surprised. 

The initial commenter's last remarks regarded the omission of some of the original sample. Apparently, some of the participants were suspicious and were removed from analyses. I am assuming that the authors meant to state that the participants in question were ones who guessed the hypothesis and hence would have been suspicious in that sense. If they were "suspicious" in any other way, that should be specified. 

The lead author did reply once to the initial critique. I will definitely buy the contention that space limitations in journals can lead to some unfortunate decisions when it comes to what to include and exclude from the narrative. Fair enough. If you've published, you've probably experienced something similar. We get it. I did feel that the lead author more or less glossed over the initial commenter's concerns and instead focused on how "rigorous" and "replicable" the study was. The initial commenter did not buy it. Nor would I. That lead author's reply seemed almost defensive, which is understandable - none of us like to be told that we might have made some mistakes that need to be corrected. 

As of this date, I have not seen an erratum or corrigendum for this article, so I am guessing that after the lead author more or less dismissed the initial concerns, there was no further escalation. That's a shame. I would have had a lot more confidence in the findings if the authors had either corrected any errors or demonstrated tangibly that any concerns were not warranted. Hey, the actual data set and the analysis plan would have gone a long way to sorting this all out. So far? Crickets.

As the late Kurt Vonnegut would say, "so it goes."

Wednesday, May 7, 2025

A milestone of sorts

It appears that as of today, this humble blog has had 250,000 visitors. It only took all of 13 years or so for that to happen, but then again, I am a relatively obscure researcher so I am not complaining. Quite the contrary: I am grateful. Thanks for dropping by and entrusting me to provide as honest and accurate account of my specialty area as is humanly possible. I'm not going anywhere. I'll have more in the upcoming months and years.

Tuesday, May 6, 2025

Postscript to the preceding

I have been spending a bit more time looking at the Replications and Reversals site I mentioned yesterday. It's worth the time. Of course I look for anything that is in my area of specialization, so I was heartened to see what strikes me as an accurate take on the weapons priming effect:


 I will note that the meta-analysis I was lead author on is reported generally fairly, but I would add that if one looked at behavioral outcomes only, that effect size is even lower, and when publication bias is factored in, depending on the method used to detect potential publication bias, the effect size is pretty much zero. That said, referring to the evidence for the weapons priming effect as mixed is a reasonable assessment. At minimum, the effect is indeed smaller than what was once believed.

Monday, May 5, 2025

Replications and Reversals

A few years ago, I wrote about a blog post documenting reversals in psychology. In the intervening years, it has turned into a full-fledged site hosted by FORRT (Replications and Reversals), and is quite comprehensive. If you are interested in what classic research has held up and what has at best mixed evidence or has been debunked, this is a valuable resource. Every time I redo my social psychology course, this is a site that will give me more ammo to make sure that my students have the most accurate information available.

Wednesday, April 23, 2025

Do violent video games cause an increase in violent crimes?

This is something I will come back to once time permits. I do want to highlight a paper I perused recently that examines the question of whether or not video games with violent and mature content increase violent crime. Basically, the authors find no such link. I am not surprised. It's one thing to make the claim that there may be a link between violent video games and mild aggression in the lab, and there has been and will continue to be some conversation about what those findings in the aggregate mean. It's quite a leap of faith to make sweeping statements about that body of research providing evidence that violent video games are responsible for real life violence, and it's a leap of faith I am not willing to make given the dearth of evidence.

Tuesday, March 25, 2025

Back to Zimbardo's Stanford Prison Experiment

A few months ago, I saw an article on the Stanford Prison Experiment, discussed it on this blog in a brief post in which I said I wanted to circle back to this. Thankfully Retraction Watch gave me an excuse, and some down time at a conference is allowing me time to say a few more words. Zimbardo's Stanford Prison Experiment (SPD going forward) was controversial from the start, as any of us with even a cursory awareness of social psychological research would know. Regrettably, it has been taken as the gospel in many textbooks (Introductory Psych textbooks as well as Social Psych textbooks) leading to the impression that the SPD was instrumental in establishing the importance of the power of the situation. Zimbardo himself certainly used the notoriety of the SPD for self-promotion throughout the remainder of his career and life. 

The Retraction Watch article I mentioned above addresses a question that seems reasonable to ask: should Zimbardo's papers on SPD be retracted? Certainly, as the guest author notes, there are good reasons to do so. First up the author considers the scientific credentials of the study. There is no well-defined a-priori hypothesis which itself is a big no-no. Where was the control group? There was none. Any tests of significance? Who needs those, right? Then there is the little matter of Zimbardo's expertise. He had no known background in criminology as I understand it. One major ethical principle is competence. Does the PI have the background to adequately design and execute the study? Can the PI adequately train their research assistants to carry out their duties in the lab? In this case, Zimbardo fails the competency test.

There is also the question of the originality of the study's design. Granted, in the sciences we build on the work of others. So as far as novelty goes, I am not that much of a stickler. But it is important to credit the sources that inspired one's work. In the case of the SPD, apparently Zimbardo got the idea from a student term paper that remained uncredited.

The ethical treatment of participants is certainly worth weighing as well. The conditions the "prisoners" in this simulation experienced were inhumane to put it bluntly. Humiliation, unsanitary conditions, and sleep deprivation were all part of the experience. The SPD was a lawsuit waiting to happen. 

I have covered the credibility of the findings elsewhere on this blog. There was certainly evidence of fraud in the reporting. Important details were left out intentionally. The seemingly "spontaneous" scene during the six days the simulation ran were effectively stage managed by Zimbardo. Maybe I am being charitable calling this the psychology of community theatre, but it captures the essence.

So all of the above would make any paper published ripe for retraction. But there is a problem. This research is over 50 years old. One article that was published in the early 1970s appeared in a journal that technically no longer exists. Retraction would be impossible in that case. 

If retraction is impossible, what is the remedy? I think I am largely in agreement with the author that the SPD is best covered in textbooks, seminars, and public media coverage as a cautionary tale of how not to conduct research or how to report findings. The study had so many ethical red flags after all. And really this is my clarion call to textbook authors: if you are working on a new book for students or updating your own, and you cover Zimbardo's SPD, make sure to treat it not as some amazing revelation about human behavior but rather as a flawed and fraudulent study that violated fundamental ethical principles. Don't leave those of us who teach Introductory Psych courses or Social Psych courses do damage control individually. 

Wednesday, January 1, 2025

Happy New Year!

 

After a bit of a break, it is time to jump right in. The academic world can be a challenging space, depending on where one lives. Even if we take location out of the equation, there are plenty of political interest groups pressuring colleges and universities to crack down on whole degree programs, faculty speech, and student speech. I may not be a free speech absolutist, but I do generally err on the side of free expression, as the academy writ large tends to function better when faculty and students can speak their minds, with the caveat that there is an agreement on the fundamentals (what I call data or what everyone else would call facts or premises). So far, I've been pretty lucky, but I always know I am always one legislative session away from running out of luck. But I choose to be hopeful. I do know we will face some serious headwinds depending on what the next president of the US pursues in terms of student visa availability, the would Department of Education, and so on. In a sense, it is a good thing that policies governing curriculum are devolved to the states and in my sector often to college or university systems. Without some centralized means of enforcing faculty or students to follow a particular party line, the odds of pulling that feat off are very minimal, but not non-existent. If we're really fortunate, the incoming US president will prove to be about as lazy and ineffective as he was during his first term (2017-2021). 

This is an especially interesting time to be studying authoritarianism. I study authoritarianism from a psychological perspective as opposed to studying authoritarian political organizations and systems. I do think the work I and others who are far more well-known does inform how to understand how an authoritarian regime could become established and perpetuated, and that those in political science and history can inform us psychologists on how those systems might influence authoritarian attitudes among the citizenry. 

I have slowed down my research activities considerably these last few years. Initially, the pandemic threw a monkey wrench into my plans. Then it just came down to a series of personal family crises these last couple of years that have kept me out of the game. Thankfully, I am now essentially a senior academic, so perhaps it matters a bit less if I publish or not. I have been revisiting a couple topics recently and should be sharing a bit of that with you at some point later on in the year. 

In the meantime, happy new year!