Both in graduate school and in my post-grad school professional life, I have periodically seen the dark underbelly of the psychological sciences. I have witnessed questionable research practices, and at times been something of a participant. On those occasions, the experience became soul-draining, for lack of a better term. In some fundamental sense, I am still trying to figure out how to recover.
I think it is important to realize that there is science as it is idealized in textbooks and science as it is actually practiced. Although not mutually exclusive, they are quite distinct. My particular area of the sciences, social psychology, as actually practiced, leaves a lot to be desired. P-hack, HARKing, hiding inconvenient data sets, and hastily composed manuscripts that may be so fundamentally flawed that they should never make it to peer review are far closer to the norm than they should be. If you are a grad student, or an obscure professor at institutions that none of the big names has even heard of or cares about, you will likely be pressured into engaging in such behavior, or looking the other way while those who believe they hold your fate in their hands.
Peer review, too, is not quite the defense against poor research it is portrayed to be in most college textbooks. The peer review system appears to me to be stretched beyond its limits. The upload portals for manuscript submission are themselves potentially powerful tools to detect some serious problems (such as duplicate content), but those are often not utilized. Elsevier's Evise platform, for example, may be used to detect duplicate content, but it also may not be used to do so. That was something disclosed to me unwittingly while I was corresponding with an associate publisher responsible for a journal called Current Opinion in Psychology. I don't think she intended to let that slip, but there it is. So the one feature that distinguishes Evise from being little more than Adobe Acrobat Reader DC (which can convert files from Word to PDF and merge multiple documents into one document) is unlikely to be utilized. Unless you see explicit language otherwise, assume that the major publishing conglomerates are not checking for plagiarism. If you are an author or peer reviewer, you are well and truly on your own. Peer review also varies in terms of the time reviewers are allowed to examine a manuscript. Most will give you a month or two. Some - usually predatory journals, but also occasionally journals that are supposed to be reputable (looking at you, Current Opinion in Psychology) - expect much quicker turnaround time (e.g., perhaps a couple days). That should worry all of us.
The academic world, from the upper echelons down to the regional colleges and universities, pressures faculty and grad students alike to adopt a publish or perish mindset. The bottom line is product, and specifically product in high impact journals. Note that high impact does not necessarily equal high quality, as even a cursory scan of retractions and corrections in these outlets will make painfully obvious. So why do we bother? High impact equals high prestige, and high prestige is what earns a first job, or a promotion and tenure. As long as you're pumping out product, regardless of other demands on your time at work, you'll be deemed productive. Those who don't may well stagnate or find themselves elbowed out of the academic world altogether. Suffice it to say, the work-life balance of many academicians is atrocious. I know, because I have lived the dream, and it may well have nearly killed me had I not walked away when I did. Note that I did not walk away from a job, but rather a mentality that turns out to be unhealthy.
From time to time, I am going to go into some serious detail about how the research you consume in textbooks, the mass media, or even directly from the original source material itself, is made. I am going to do so because not only is there a need to try to advocate for a better science of psychology than currently exists, but because this is also very personal. I walked away from a collaborative relationship earlier this spring. I am hardly a victim in what happened over a three year period. However, the consequences of allowing that situation to go on for as long as it did turned out to be damaging both physically and psychologically. Had I allowed things to continue, I have little doubt I would have continued down a path that would have not only destroyed my career, but ultimately ended my life. I note that not to be overly dramatic, but merely to highlight that the level of toxicity that exists in my field does inflict irreparable damage. Only now do I feel like I am even modestly recovering.
Just as one of my intellectual heroes, Anthony Bourdain would argue against ordering a steak well-done, I am going to argue against consuming substandard research. I will do so by telling my story, and also by giving you some tools that you can use to help critically evaluate what you read. In the meantime, stay tuned. I am just getting warmed up.
No comments:
Post a Comment