Interesting post by Gelman recently. I am on the mailing list from where the quoted email came. It was in reference to revelations about the Zimbardo prison experiment that cast further doubt on its legitimacy. As someone watching HBO's Chernobyl series, there is something almost Soviet in the mindset expressed in that email. The thing about clampdowns is that they tend to generate further cynicism that erodes the edifice upon which a particular discipline or sub-discipline is based. If I could, I'd tell these folks that they are just making the changes they are fighting more inevitable, even if for a brief spell the life of those labeled as dissidents is made a bit more inconvenient.
The title for this and Gelman's post is inspired by a song by The Clash:
Tuesday, May 28, 2019
Sunday, May 19, 2019
A Little Matter of Data Quality
A quote from Andrew Gelman:
If you have followed my blog over the last few months, you have an idea of what I've been going on about, yeah? Numbers mean squat if I cannot trust their source. Think about that the next time someone gives you an improbably claim, offers up some very complex looking tables, figures, and test statistics, and then hopes you don't notice that the tables are a bit odd, the marginal means and cell means don't quite mesh they way they should, or that there were serious decision errors. Beware especially of work coming from researchers who are unusually prolific at publishing findings utilizing methods that would take heroic team efforts to publish at that rate, let alone a single individual. Garbage data give us garbage findings more often than not. Seems like a safe enough bet.
I go on about this because there is plenty of dodgy work in my field. There is reason to be concerned about some of the zombies (i.e., phenomena that should have been debunked that continue to be taught and treated as part of our popular lore) in my field. Stopping the proliferation of these zombies at this point is a multifaceted effort. Part of that effort is making sure we can actually examine the data from which findings are derived. In the meantime, remember rule #2 for surviving a zombie apocalypse (including zombie concepts): the double tap.
So it’s good to be reminded: “Data” are just numbers. You need to know where the data came from before you can learn anything from them.
If you have followed my blog over the last few months, you have an idea of what I've been going on about, yeah? Numbers mean squat if I cannot trust their source. Think about that the next time someone gives you an improbably claim, offers up some very complex looking tables, figures, and test statistics, and then hopes you don't notice that the tables are a bit odd, the marginal means and cell means don't quite mesh they way they should, or that there were serious decision errors. Beware especially of work coming from researchers who are unusually prolific at publishing findings utilizing methods that would take heroic team efforts to publish at that rate, let alone a single individual. Garbage data give us garbage findings more often than not. Seems like a safe enough bet.
I go on about this because there is plenty of dodgy work in my field. There is reason to be concerned about some of the zombies (i.e., phenomena that should have been debunked that continue to be taught and treated as part of our popular lore) in my field. Stopping the proliferation of these zombies at this point is a multifaceted effort. Part of that effort is making sure we can actually examine the data from which findings are derived. In the meantime, remember rule #2 for surviving a zombie apocalypse (including zombie concepts): the double tap.
Monday, May 13, 2019
Causus Belli
I know I have been hard on work from Qian Zhang's lab in Southwest University for a while now. I have my reasons. I am mainly concerned with a large number of papers in which there are many serious errors. I can live with the one-off bad article. That happens. What I am reading suggests either considerable incompetence over multiple studies or something far more serious. There is a pervasive pattern of errors that is consistent across multiple articles. That I or any of my colleagues were stonewalled when asking for data is not acceptable given the pattern of results over the course of the last several years. My recent experiences have turned me from a "trust in peer review" to "trust but verify." If verification fails, trust goes bye-bye. Just how it is.
Given the sheer quantity of articles and given the increasing level of impact each new article has, I have good cause to be concerned. I am even more concerned given that well-known American and European authors are now collaborators in this research. They have reputations on the line, and the last thing I want for them is to find themselves dealing with corrections and retractions. Beyond that, I can never figure out how to say no to a meta-analysis. The findings in this body of research are ones that I would ordinarily need to include. As of now, I am questioning if I could even remotely hope to extract accurate effect sizes from this particular set of articles. I should never find myself in that position, and I think that anyone in such a position is right to be upset.
Under ordinary circumstances, I am not a confrontational person. If anything, I am quite the opposite. However, when I see something that is just plain wrong, I cannot remain silent. There is a moral and ethical imperative for speaking out. Right now I see a series of articles that have grave errors, and ones in which would lead a reasonable skeptic to state that the main effect the authors sought (weapons priming, video game priming, violent media priming) never existed. There may or may not be some subset effect going on, but without the ability to reproduce the original findings, there is no way to know entirely for sure. Not being able to trust what I read is extremely uncomfortable. I can live with uncertainty - after all a certain level of uncertainty is built into our research designs and our data analysis techniques. What I cannot live with is no certainty at all. To take an old John Lennon song (that I probably knew more from an old Generation X cover), "gimme some truth." Is that too much to ask? If so, I continue to be confrontational.
Given the sheer quantity of articles and given the increasing level of impact each new article has, I have good cause to be concerned. I am even more concerned given that well-known American and European authors are now collaborators in this research. They have reputations on the line, and the last thing I want for them is to find themselves dealing with corrections and retractions. Beyond that, I can never figure out how to say no to a meta-analysis. The findings in this body of research are ones that I would ordinarily need to include. As of now, I am questioning if I could even remotely hope to extract accurate effect sizes from this particular set of articles. I should never find myself in that position, and I think that anyone in such a position is right to be upset.
Under ordinary circumstances, I am not a confrontational person. If anything, I am quite the opposite. However, when I see something that is just plain wrong, I cannot remain silent. There is a moral and ethical imperative for speaking out. Right now I see a series of articles that have grave errors, and ones in which would lead a reasonable skeptic to state that the main effect the authors sought (weapons priming, video game priming, violent media priming) never existed. There may or may not be some subset effect going on, but without the ability to reproduce the original findings, there is no way to know entirely for sure. Not being able to trust what I read is extremely uncomfortable. I can live with uncertainty - after all a certain level of uncertainty is built into our research designs and our data analysis techniques. What I cannot live with is no certainty at all. To take an old John Lennon song (that I probably knew more from an old Generation X cover), "gimme some truth." Is that too much to ask? If so, I continue to be confrontational.
Thursday, May 9, 2019
A word about undergraduate research projects
I suppose standards about acceptable practices for undergraduate research projects vary from institution to institution and across countries. I do have a few observations of my own, based on an admittedly very, very small sample of institutions in one country.
I have worked at a university where we had the staffing and resources to give students introductory stats training and an introductory methods course - the latter usually taken during the Junior year. None of those projects was ever intended for publication, given the small samples involved, and given that students were expected to produce a finished research project in one semester. At my current university, my department requires students go through an intensive four-course sequence of statistical and methodological training. Students are required to learn enough basic stats to get by, learn how to put together a research prospectus, and then gain some more advanced training (including creating and managing small databases) in statistics and in methodology. The whole sequence culminates with students presenting their finished research on campus. That may seem like a lot, but by the time students are done, they have at least the basics required to handle developing a thesis prospectus in grad school. At least they won't do what I did and ask, "what is a prospectus?" That was a wee bit embarrassing.
Each fall semester, I write a general IRB proposal to cover most of these projects for my specific course sections. That IRB proposal is limited to a specific on-campus adult sample and to minimal risk designs. None of those projects covered under that general IRB proposal are intended for publication. Students wanting to go the extra mile need to complete their own IRB forms and gain approval. Students who are genuinely interested in my area of expertise go through the process of completing their own IRB proposals and dealing with any revisions, etc., before we even think of running anything.
Only a handful of those have produced anything that my students wished to pursue publication. To date, I have one successfully published manuscript with an undergrad, one that was rejected (it was an interesting project, but admittedly the sample was small and findings too inconclusive), and one that is currently under review. That these students were coauthors means that they contributed significantly to the writeup. That means my peers could grill them at presentation time and they could give satisfactory answers. They knew their stuff. And the reason they knew their stuff is because I went out of my way to make sure that they were mentored as they made those projects their own. I made sure that I had seen the raw data and worked together with each student to make sure data were analyzed correctly. I stick to fairly simple to accomplish personality-social projects in those cases as that is my training. That's just how we roll.
I have worked at a university where we had the staffing and resources to give students introductory stats training and an introductory methods course - the latter usually taken during the Junior year. None of those projects was ever intended for publication, given the small samples involved, and given that students were expected to produce a finished research project in one semester. At my current university, my department requires students go through an intensive four-course sequence of statistical and methodological training. Students are required to learn enough basic stats to get by, learn how to put together a research prospectus, and then gain some more advanced training (including creating and managing small databases) in statistics and in methodology. The whole sequence culminates with students presenting their finished research on campus. That may seem like a lot, but by the time students are done, they have at least the basics required to handle developing a thesis prospectus in grad school. At least they won't do what I did and ask, "what is a prospectus?" That was a wee bit embarrassing.
Each fall semester, I write a general IRB proposal to cover most of these projects for my specific course sections. That IRB proposal is limited to a specific on-campus adult sample and to minimal risk designs. None of those projects covered under that general IRB proposal are intended for publication. Students wanting to go the extra mile need to complete their own IRB forms and gain approval. Students who are genuinely interested in my area of expertise go through the process of completing their own IRB proposals and dealing with any revisions, etc., before we even think of running anything.
Only a handful of those have produced anything that my students wished to pursue publication. To date, I have one successfully published manuscript with an undergrad, one that was rejected (it was an interesting project, but admittedly the sample was small and findings too inconclusive), and one that is currently under review. That these students were coauthors means that they contributed significantly to the writeup. That means my peers could grill them at presentation time and they could give satisfactory answers. They knew their stuff. And the reason they knew their stuff is because I went out of my way to make sure that they were mentored as they made those projects their own. I made sure that I had seen the raw data and worked together with each student to make sure data were analyzed correctly. I stick to fairly simple to accomplish personality-social projects in those cases as that is my training. That's just how we roll.
Vive La Fraud?
Hang on to your hats, folks. This is just wild.
If you've never read about Nicholas Guéguen before, read this post in Retraction Watch. Nick Brown and James Heathers go into far more detail in their own blog post.
Guéguen is certainly prolific - a career that spans a couple decades has yielded some 336 published articles. He also has published some books, although I am not particularly familiar with the publisher involved. It's not that the topics of interest are a bit out of the ordinary. Spend enough time in Psychology or any related science for any length of time and you'll find someone researching something that will make you wonder. However, as long as the methods are legit and the research can withstand independent replication efforts, I would hope most of us would accept that the phenomena under investigation are at least apparently real and potentially of interest to some subset of the human species. Sort of is what it is.
Rather, it is the methodology that Guéguen employs that causes concern. In particular, his field research apparently is conducted by beginning research methods students - the vast majority of whom have a minimal grasp of what they are doing - and then published under his own name as if the work were his own (students were apparently never told that this might occur). Worse, at least one student I am aware of, based on Brown and Heathers' work, owned up to the apparent reality that students tended to fabricate results that were turned in for a grade in Guéguen's methods courses over the years. Whether or not Guéguen was aware of that is certainly a worthy question to ask. At minimum, I agree with Brown and Heathers that this is a body of research that deserves careful scrutiny.
Supposedly, two of the articles in question were supposed to have been retracted by now, but apparently are not. My limited experience with editors is that when one is the bearer of bad tidings, the tendency is to ignore for as long as possible, drag their heels even longer, and hope the problem (i.e., legitimate complaints about dodgy research) goes away. Some other things to remember - editors and publishers never make mistakes - according to editors and publishers. When the proverbial truth hits the fan, their legal beagles will provide them with whatever cover is needed to avoid accountability. Regardless, expect a situation like this one to drag on for a while. Heck, it took Markey and Elson how many years to get the original Boom! Headshot! article retracted? I am guessing that in the case of some articles I have been following, it will easily be a couple years before anything even remotely resembling satisfaction occurs. Once things go adversarial, that's just the way it is - and our incentive systems reward being adversarial. The only time a retraction might go relatively quickly (as in a few months) is if you get that one author in a blue moon who hollers at an editor and says "dude, I really made a mess of things - do something." If you find yourself in that situation (try to avoid, please), do save your email correspondence with any editor and associate publishers, publishers, etc. Document everything. You'll be glad you did.
If you've never read about Nicholas Guéguen before, read this post in Retraction Watch. Nick Brown and James Heathers go into far more detail in their own blog post.
Guéguen is certainly prolific - a career that spans a couple decades has yielded some 336 published articles. He also has published some books, although I am not particularly familiar with the publisher involved. It's not that the topics of interest are a bit out of the ordinary. Spend enough time in Psychology or any related science for any length of time and you'll find someone researching something that will make you wonder. However, as long as the methods are legit and the research can withstand independent replication efforts, I would hope most of us would accept that the phenomena under investigation are at least apparently real and potentially of interest to some subset of the human species. Sort of is what it is.
Rather, it is the methodology that Guéguen employs that causes concern. In particular, his field research apparently is conducted by beginning research methods students - the vast majority of whom have a minimal grasp of what they are doing - and then published under his own name as if the work were his own (students were apparently never told that this might occur). Worse, at least one student I am aware of, based on Brown and Heathers' work, owned up to the apparent reality that students tended to fabricate results that were turned in for a grade in Guéguen's methods courses over the years. Whether or not Guéguen was aware of that is certainly a worthy question to ask. At minimum, I agree with Brown and Heathers that this is a body of research that deserves careful scrutiny.
Supposedly, two of the articles in question were supposed to have been retracted by now, but apparently are not. My limited experience with editors is that when one is the bearer of bad tidings, the tendency is to ignore for as long as possible, drag their heels even longer, and hope the problem (i.e., legitimate complaints about dodgy research) goes away. Some other things to remember - editors and publishers never make mistakes - according to editors and publishers. When the proverbial truth hits the fan, their legal beagles will provide them with whatever cover is needed to avoid accountability. Regardless, expect a situation like this one to drag on for a while. Heck, it took Markey and Elson how many years to get the original Boom! Headshot! article retracted? I am guessing that in the case of some articles I have been following, it will easily be a couple years before anything even remotely resembling satisfaction occurs. Once things go adversarial, that's just the way it is - and our incentive systems reward being adversarial. The only time a retraction might go relatively quickly (as in a few months) is if you get that one author in a blue moon who hollers at an editor and says "dude, I really made a mess of things - do something." If you find yourself in that situation (try to avoid, please), do save your email correspondence with any editor and associate publishers, publishers, etc. Document everything. You'll be glad you did.
Tuesday, May 7, 2019
Gardening With the Professor
I have been dealing with some seriously heavy stuff as of late and in need of diversions. Finally I am finding some excuse to get away from the computer and away from scrutinizing research long enough to do something a bit more physical.
When I put a down payment on my current digs, the previous owners had a flower bed in the front yard that had some bushes, but for the most part had placed these fake plastic bushes with fake flowers around that flower bed. The realtor apparently thought that was a good idea for staging the house - the fake plants added color. Fair enough. But those do degrade over time. So too do some of the real plants. And since I live right along the western edge of North America's Eastern Forest, I get some unwanted plants in that flower bed. To give you some idea, last fall I noticed that two trees had taken root in the flower bed. Those were way too close to the house and would have done structural damage eventually. So I cut those down to stumps right before the Winter Solstice with the idea of removing the stumps and replacing them with more proper flowering plants.
About a day ago, I went by a home supply store with a nursery, bought a couple plants that I liked and brought them home. I made a mental note of where I wanted them placed. Late the following afternoon, I come home from work and errands. Of course I pick the hottest day of the calendar year, so far, to do some grueling physical work, but that is okay. Those tree stumps still needed to be dealt with, especially since I had not quite succeeded in killing those off late last fall. I cut off new limbs that were forming, and then went to work digging up the two stumps and their root systems. They were thankfully small enough trees at this point to where I could do that without having to call someone and pay for their services. Once the stumps and roots had been sufficiently removed, I went to work digging the holes where I wanted my new plants. I got both of them in place, covered the base of each with sufficient dirt, cleared up the debris and my tools. We'll just say that by the time I was done, I was definitely ready for a shower and plenty of hydration. I also made sure my new plants had some hydration.
I think they look lovely. They are Veronica "First Love" plants. The flowers are an almost flourescent violet-pink. They require partial to full sunlight and at least weekly watering. Where I live, both are doable. Hopefully they do well. In the next year or two, I will have to decide what I want to do about the rose bush in the flower bed. It is showing its age. However, it is still producing some lovely red roses. This spring has been one of its best for yielding roses. I have contemplated eventually replacing it and some boxwoods elsewhere on the property with new rose plants. I don't think you can have too many roses. Of course I will have to hire a contractor to remove the older plants. They're too established. The rest of the work should go easily enough. I'll give that a year or two. Want to save a few pennies first.
Last year, I re-established bulb plants in a back yard flower bed - including, of course tulips. There are a couple other flower beds that went bare a while back. I am still contemplating what to do with that area. I am thinking perhaps sunflowers. We've had sunflowers in the past. They are an annual plant, and usually I prefer perennials. But they are quite nice to look at, and although not completely dog-proof, they seem to do okay. We get some honey bees and bumble bees around the property, and I would like to encourage more of our pollenators to do the work they were evolved to do.
Hopefully I will have some more progress before long.
When I put a down payment on my current digs, the previous owners had a flower bed in the front yard that had some bushes, but for the most part had placed these fake plastic bushes with fake flowers around that flower bed. The realtor apparently thought that was a good idea for staging the house - the fake plants added color. Fair enough. But those do degrade over time. So too do some of the real plants. And since I live right along the western edge of North America's Eastern Forest, I get some unwanted plants in that flower bed. To give you some idea, last fall I noticed that two trees had taken root in the flower bed. Those were way too close to the house and would have done structural damage eventually. So I cut those down to stumps right before the Winter Solstice with the idea of removing the stumps and replacing them with more proper flowering plants.
About a day ago, I went by a home supply store with a nursery, bought a couple plants that I liked and brought them home. I made a mental note of where I wanted them placed. Late the following afternoon, I come home from work and errands. Of course I pick the hottest day of the calendar year, so far, to do some grueling physical work, but that is okay. Those tree stumps still needed to be dealt with, especially since I had not quite succeeded in killing those off late last fall. I cut off new limbs that were forming, and then went to work digging up the two stumps and their root systems. They were thankfully small enough trees at this point to where I could do that without having to call someone and pay for their services. Once the stumps and roots had been sufficiently removed, I went to work digging the holes where I wanted my new plants. I got both of them in place, covered the base of each with sufficient dirt, cleared up the debris and my tools. We'll just say that by the time I was done, I was definitely ready for a shower and plenty of hydration. I also made sure my new plants had some hydration.
I think they look lovely. They are Veronica "First Love" plants. The flowers are an almost flourescent violet-pink. They require partial to full sunlight and at least weekly watering. Where I live, both are doable. Hopefully they do well. In the next year or two, I will have to decide what I want to do about the rose bush in the flower bed. It is showing its age. However, it is still producing some lovely red roses. This spring has been one of its best for yielding roses. I have contemplated eventually replacing it and some boxwoods elsewhere on the property with new rose plants. I don't think you can have too many roses. Of course I will have to hire a contractor to remove the older plants. They're too established. The rest of the work should go easily enough. I'll give that a year or two. Want to save a few pennies first.
Last year, I re-established bulb plants in a back yard flower bed - including, of course tulips. There are a couple other flower beds that went bare a while back. I am still contemplating what to do with that area. I am thinking perhaps sunflowers. We've had sunflowers in the past. They are an annual plant, and usually I prefer perennials. But they are quite nice to look at, and although not completely dog-proof, they seem to do okay. We get some honey bees and bumble bees around the property, and I would like to encourage more of our pollenators to do the work they were evolved to do.
Hopefully I will have some more progress before long.
Still a bit strange
Another article from the Zhang lab was published very recently. I do have to say that the format of the manuscript is well-done compared to many of the earlier papers. There are some initial concerns I will voice now. I may come back to this one later when and if the moment arises.
The literature review notes a number of meta-analyses that purport to provide support for media violence causing aggressive outcomes. The authors do offer a quick summary of several other meta-analyses that show that the average effect sizes from media violence research are negligible. Then the authors quickly state that the evidence is "overwhelming" that violent media leads to aggression. Um....not quite. There is actually a serious debate in which the extent that any link between exposure to violent content in films, video games, etc. and actual aggressive behavior, and the impression I get is that at best the matter is far from settled. The evidence for is arguably underwhelming rather than overwhelming. But hey, let's blow through all that research and at least check off the little box saying it was cited before discarding its message.
I am not too keen on the idea of throwing participants out of a study unless there is a darned good reason. Equipment malfunctions, failures to follow instructions, suspicion (i.e., guessing the hypothesis) would strike me as good reasons. Merely trying to get an even number of participants in treatment and control condition is not in and of itself a good reason. If one is inclined to do so anyway and state that the participants whose data were examined were randomly chosen, then at least go into some detail as to what that procedure entailed.
I will admit that I do not know China particularly well, but I am a bit taken aback that 15 primary schools could yield over 3,000 kids who are exactly 10 years of age. That is...a lot. Those schools must be huge. Then again, these kids go to school in a mega-city, so perhaps this is within the realm of possibility. This is one of those situations where I am a bit on the skeptical side, but I won't rule it out. Research protocols would certainly clarify matters on that point.
I am not sure why the authors use Cohen's d for effect size estimates for main effect analyses and then use eta square for the remaining ANOVA analyses. Personally I would prefer consistency. It's those inconsistencies that make me want to ask questions. At some point I will dive deeper into the mediation analyses. Demonstrating that accessibility of aggressive thoughts mediates the link between a particular exemplar of violent media and aggression is the great white whale that aggression researchers have been trying to chase for a good while now. If true and replicable, this would be some potentially rare good news for models of aggression derived from a social cognition perspective.
It is not clear if there were any manipulation checks included in the experimental protocols, nor if there was any extensive debriefing for suspicion - i.e. hypothesis guessing. In reaction time experiments that I ran, as well as any experiments using the competitive reaction time as a measure of aggression, it was standard operating procedure to not only have manipulation checks and an extensive debriefing with each participant, as problems like suspicion could contaminate the findings. Maybe those are procedural practices that have been abandoned altogether? I would hope not.
One of the most difficult tasks in conducting any media violence experiment is ascertaining that the violent and nonviolent media samples in question are as equivalent as possible except of course level of violent content. It is possible that the cartoon clips the authors use are perfectly satisfactory. Unfortunately we have to take that on faith for the time being.
At the end of the day, I am left with a gut feeling that I shouldn't quite believe what I am reading, even if it appears relatively okay on the surface. There are enough holes in the report itself that I suspect a well-versed skeptic can have a field day. Heck, as someone who is primarily an educator, I am already finding inconvenient questions and all I have done is give the paper an initial reading. This is my hot take on this particular paper.
Assuming the data check out, what I do appear to be reading thus far suggests that these are effects that are small enough to where I would not want to write home about them. In other words, in a best case scenario, I doubt this paper is going to be the one to change any minds. It will appeal to those needing to believe that violent content in various forms of mass media are harmful, and it will be either shrugged off or challenged by skeptics. I guess this was good enough for peer review. It is what it is. As I have stated elsewhere, peer review is a filter, and not a perfect filter. The rest is left to us who want to tackle research post-peer review.
The literature review notes a number of meta-analyses that purport to provide support for media violence causing aggressive outcomes. The authors do offer a quick summary of several other meta-analyses that show that the average effect sizes from media violence research are negligible. Then the authors quickly state that the evidence is "overwhelming" that violent media leads to aggression. Um....not quite. There is actually a serious debate in which the extent that any link between exposure to violent content in films, video games, etc. and actual aggressive behavior, and the impression I get is that at best the matter is far from settled. The evidence for is arguably underwhelming rather than overwhelming. But hey, let's blow through all that research and at least check off the little box saying it was cited before discarding its message.
I am not too keen on the idea of throwing participants out of a study unless there is a darned good reason. Equipment malfunctions, failures to follow instructions, suspicion (i.e., guessing the hypothesis) would strike me as good reasons. Merely trying to get an even number of participants in treatment and control condition is not in and of itself a good reason. If one is inclined to do so anyway and state that the participants whose data were examined were randomly chosen, then at least go into some detail as to what that procedure entailed.
I will admit that I do not know China particularly well, but I am a bit taken aback that 15 primary schools could yield over 3,000 kids who are exactly 10 years of age. That is...a lot. Those schools must be huge. Then again, these kids go to school in a mega-city, so perhaps this is within the realm of possibility. This is one of those situations where I am a bit on the skeptical side, but I won't rule it out. Research protocols would certainly clarify matters on that point.
I am not sure why the authors use Cohen's d for effect size estimates for main effect analyses and then use eta square for the remaining ANOVA analyses. Personally I would prefer consistency. It's those inconsistencies that make me want to ask questions. At some point I will dive deeper into the mediation analyses. Demonstrating that accessibility of aggressive thoughts mediates the link between a particular exemplar of violent media and aggression is the great white whale that aggression researchers have been trying to chase for a good while now. If true and replicable, this would be some potentially rare good news for models of aggression derived from a social cognition perspective.
It is not clear if there were any manipulation checks included in the experimental protocols, nor if there was any extensive debriefing for suspicion - i.e. hypothesis guessing. In reaction time experiments that I ran, as well as any experiments using the competitive reaction time as a measure of aggression, it was standard operating procedure to not only have manipulation checks and an extensive debriefing with each participant, as problems like suspicion could contaminate the findings. Maybe those are procedural practices that have been abandoned altogether? I would hope not.
One of the most difficult tasks in conducting any media violence experiment is ascertaining that the violent and nonviolent media samples in question are as equivalent as possible except of course level of violent content. It is possible that the cartoon clips the authors use are perfectly satisfactory. Unfortunately we have to take that on faith for the time being.
At the end of the day, I am left with a gut feeling that I shouldn't quite believe what I am reading, even if it appears relatively okay on the surface. There are enough holes in the report itself that I suspect a well-versed skeptic can have a field day. Heck, as someone who is primarily an educator, I am already finding inconvenient questions and all I have done is give the paper an initial reading. This is my hot take on this particular paper.
Assuming the data check out, what I do appear to be reading thus far suggests that these are effects that are small enough to where I would not want to write home about them. In other words, in a best case scenario, I doubt this paper is going to be the one to change any minds. It will appeal to those needing to believe that violent content in various forms of mass media are harmful, and it will be either shrugged off or challenged by skeptics. I guess this was good enough for peer review. It is what it is. As I have stated elsewhere, peer review is a filter, and not a perfect filter. The rest is left to us who want to tackle research post-peer review.
Monday, May 6, 2019
Patterns
The lab of which I have posted over intensively over the last few weeks is nothing if not consistent. We could even say that the lab has a sort of assembly line for doing research. The basic research design barely changes. The only things that change are perhaps the stimuli, the participants, and occasionally the dependent variable.
If you were to look for a metaphor that I understand, let's think about auto manufacturing. It is not uncommon for a major corporate conglomerate in the auto business to have a common platform across that is shared across various brand names. Change the headlights and grill, the tail lights, and the hubcaps (and perhaps various trim levels) and you can go from a Chevy to a Cadillac, just like that. If you are not wanting to believe me, look at the offerings for SUVs by Chevy, GMC, and Cadillac. You could do the same for Ford or Chrysler products.
An assembly line approach to social-personality research is not in itself intrinsically bad. There is a certain efficiency to that approach that is very much valued in our current capitalistic system. It is an approach that provides consistency as far as expectations are concerned, and is arguably more likely to be rewarded with grant dollars. It is what it is. We could even argue that a certain expertise is built in the process. In an auto assembly line, we swap out the headlights and grill, the tail lights, and the hubcaps, and perhaps some other creature comforts in the cabin and we go from branding the final product as an entry level vehicle or something more as a luxury vehicle. Regardless, we know how it will perform, we know what to expect in terms of quality control, and in terms of resale value. Maybe that is not how the Party in China would want to publicly frame things, butt it is how our global capitalistic system works.
In the lab I have been critiquing, we can swap out the stimuli (violence in video games, violence in films, presence or absence of weapons), participants (preadolescent children, adolescents, or adult college students), and perhaps the operational definition of aggressive cognition (reaction time on a Stroop task or reaction time on some equivalent of a lexical decision task), but expect some consistency in results. In and of itself not necessarily a bad thing. Where things get tricky is in the process whenever we change one or more of the elements. My hunch for a while is that the lab in question has used a template, swapped in or out whatever needed to be swapped in or out, and reported findings as quickly as possible. Again, given the incentive system in capitalist economies, it is to be expected. Efficiency is rewarded. Inefficiency, not so much. The end result is the same. There is no honest way to mistake a Chevy for a Cadillac, no matter how hard one might try to rebrand it.
I am less concerned with a lab using a template for conducting research than I am the end result. It is one thing if there are a series of experiments that are conducted largely the same way, save for the requisite changes to IVs, samples, and DVs. It is another thing when there are some serious mistakes based on relying on a template. What if the same test statistics are found in multiple experiments? What if an earlier sample is included in a larger study without readers understanding what was done and why? What if the same errors that made Chevy products unreliable back in the day (any one recall the Chevy Cavalier?) are found in equivalent Cadillac products (for example the Cimarron)?
What I am imparting is a lesson I learned a long time ago. It is one thing to be consistent. It is another to be consistently wrong. I have been concentrating on a lab whose products have been consistently wrong for a reason. That has less to do with the lab itself than on the fact that each lab has a consistent pattern of behaviors that itself has led to non-replicable findings that have led us to our current crisis. It is very possible to be consistently wrong. Changing the way variables are measured or changing the nature of the sample does not change that reality.
The past cannot be changed. At best, we can try to correct the record. We can change how we act in the here and now, as well as in the future. As we evaluate ourselves and others, we can ask about our patterns of doing our work, and if those patterns are truly serving us well, and more importantly - are they serving our science and the public well? If I seem obsessed, it is because that is what keeps me awake at night. Arguably, I need a new hobby. In the meantime, I have questions. Those questions deserve answers. Science is not self-correcting. It takes real living and breathing human beings to ask tough questions, as well as real living and breathing human beings to make conscious choices to own up to mistakes that they have made. Doing so is hardly easy. In an ideal world, doing so would be done in a sense where some kind of forebearance were the rule. We are far from that ideal. However, that is the ideal that matters most to me.
If you were to look for a metaphor that I understand, let's think about auto manufacturing. It is not uncommon for a major corporate conglomerate in the auto business to have a common platform across that is shared across various brand names. Change the headlights and grill, the tail lights, and the hubcaps (and perhaps various trim levels) and you can go from a Chevy to a Cadillac, just like that. If you are not wanting to believe me, look at the offerings for SUVs by Chevy, GMC, and Cadillac. You could do the same for Ford or Chrysler products.
An assembly line approach to social-personality research is not in itself intrinsically bad. There is a certain efficiency to that approach that is very much valued in our current capitalistic system. It is an approach that provides consistency as far as expectations are concerned, and is arguably more likely to be rewarded with grant dollars. It is what it is. We could even argue that a certain expertise is built in the process. In an auto assembly line, we swap out the headlights and grill, the tail lights, and the hubcaps, and perhaps some other creature comforts in the cabin and we go from branding the final product as an entry level vehicle or something more as a luxury vehicle. Regardless, we know how it will perform, we know what to expect in terms of quality control, and in terms of resale value. Maybe that is not how the Party in China would want to publicly frame things, butt it is how our global capitalistic system works.
In the lab I have been critiquing, we can swap out the stimuli (violence in video games, violence in films, presence or absence of weapons), participants (preadolescent children, adolescents, or adult college students), and perhaps the operational definition of aggressive cognition (reaction time on a Stroop task or reaction time on some equivalent of a lexical decision task), but expect some consistency in results. In and of itself not necessarily a bad thing. Where things get tricky is in the process whenever we change one or more of the elements. My hunch for a while is that the lab in question has used a template, swapped in or out whatever needed to be swapped in or out, and reported findings as quickly as possible. Again, given the incentive system in capitalist economies, it is to be expected. Efficiency is rewarded. Inefficiency, not so much. The end result is the same. There is no honest way to mistake a Chevy for a Cadillac, no matter how hard one might try to rebrand it.
I am less concerned with a lab using a template for conducting research than I am the end result. It is one thing if there are a series of experiments that are conducted largely the same way, save for the requisite changes to IVs, samples, and DVs. It is another thing when there are some serious mistakes based on relying on a template. What if the same test statistics are found in multiple experiments? What if an earlier sample is included in a larger study without readers understanding what was done and why? What if the same errors that made Chevy products unreliable back in the day (any one recall the Chevy Cavalier?) are found in equivalent Cadillac products (for example the Cimarron)?
What I am imparting is a lesson I learned a long time ago. It is one thing to be consistent. It is another to be consistently wrong. I have been concentrating on a lab whose products have been consistently wrong for a reason. That has less to do with the lab itself than on the fact that each lab has a consistent pattern of behaviors that itself has led to non-replicable findings that have led us to our current crisis. It is very possible to be consistently wrong. Changing the way variables are measured or changing the nature of the sample does not change that reality.
The past cannot be changed. At best, we can try to correct the record. We can change how we act in the here and now, as well as in the future. As we evaluate ourselves and others, we can ask about our patterns of doing our work, and if those patterns are truly serving us well, and more importantly - are they serving our science and the public well? If I seem obsessed, it is because that is what keeps me awake at night. Arguably, I need a new hobby. In the meantime, I have questions. Those questions deserve answers. Science is not self-correcting. It takes real living and breathing human beings to ask tough questions, as well as real living and breathing human beings to make conscious choices to own up to mistakes that they have made. Doing so is hardly easy. In an ideal world, doing so would be done in a sense where some kind of forebearance were the rule. We are far from that ideal. However, that is the ideal that matters most to me.
About those Stroop task findings (and other assorted oddities)
I am hoping you all are generally familiar with the Stroop Interference Task. It has been around a long time. If you need a refresher, this will at least be a start. In my neck of the research woods, some form of an Emotional Stroop Task is occasionally used. It works just like the original task. Individuals are presented with multiple trials in which a word is paired with a color and the individual is supposed to name the color. Of interest is reaction time to naming the color. If there is some sort of interference, individuals should respond more slowly to the stimuli than they would otherwise.
In aggression research, an example might help. Here is a version of a Stroop task in which individuals are primed with either weapons or neutral objects:
And here are the results:
Notice that in the weapon priming condition, participants took significantly longer to respond to colors paired with aggressive words than neutral words. Contrast that with those participants in the neutral objects condition.
So far so good.
Now, here is an example of a Stroop task in which the prime stimulus is movie violence. Check out the results:
Notice something odd. Presumably, if violent movies were priming aggressive thoughts, the reaction times for aggressive words would have been significantly higher than for non-aggressive words in the violent movie condition. Instead, reaction times are lower. Strange.
Of course, there is some other weirdness. We get overall mean reaction times for aggressive and non-aggressive words in Table 2. Those could not possibly be right if the means and standard deviations are correct in Table 3. Again, something is just not adding up. I could nitpick a bit and ask why there are no decimal points for the means in each cell, but there are decimal points for standard deviations. That is a very, very minor complaint. The larger complaint is that the authors are not going to be able to say that violent movies increase accessibility of aggressive thoughts based on what they presented using a Stroop task, and that there is a huge (I mean HUGE) disconnect between the overall mean reaction times in Table 2 and the cell means provided in Table 3. After that, all we get are mean difference scores. I don't necessarily have a problem with that, although in Table 5, there is no way that the cell means provided are going to square with the marginal means for violent and non-violent movie conditions. That is also problematic. And if those difference scores are to be believed, it would appear that short-term exposure to a violent movie might actually suppress accessibility of aggressive thoughts among the subsample measuring high in trait aggressiveness, which is not what the authors wish to argue.Again, that is highly problematic. I suppose that the authors made some typos when they were writing up the paper? There is no way of knowing without access to the data set.
Also, Table 4 is just odd. It is unclear just how a MANCOVA would be appropriate as the only DV that the authors consider for the remaining analyses is a difference score. MANOVA and MANCOVA are appropriate analytic techniques for situations in which multiple DVs are analyzed simultaneously. The authors fail to list a covariate. Maybe it is gender? Hard to say. Without an adequate explanation, we as readers are left to guess. Even if a MANCOVA were appropriate, Table 4 is a case study in how not to set up a MANCOVA table. Authors should be explicit about what they are doing as possible. I can read Method and Results sections just fine, thank you. I cannot, however, read minds. Zhang et al. (2013) have placed us all in the position of being mind-readers.
Again, this is a strange article in which the analyses don't make much sense. Something is off in the way the descriptive statistics are reported. Something is off in the way that the inferential statistics are reported. The findings perhaps "looked" superficially good enough to get through peer review. It is possible that depending on the set of reviewers, the findings were ones that fit their own pre-existing beliefs - in other words some form of confirmation bias. I can only speculate on that, and will simply leave it at that. Speculation is not ideal of course.
This particular journal is not exactly a premier journal, which is hardly a knock on it. A number of low-impact journals do no worse, as near as I can tell, then the more premier journals when it comes to the quality of peer review. What matters is that this is an example of an article that slipped through peer review that arguably should not have. Given that this same lab has produced a number of subsequent articles with many of the same problems in more premier journals, I am justifiably worried.
Consider this a cautionary tale.
Reference:
Zhang, Q. , Zhang, D. & Wang, L. (2013). Is Aggressive Trait Responsible for Violence? Priming Effects of Aggressive Words and Violent Movies. Psychology, 4, 96-100. doi: 10.4236/psych.2013.42013
And here are the results:
Notice that in the weapon priming condition, participants took significantly longer to respond to colors paired with aggressive words than neutral words. Contrast that with those participants in the neutral objects condition.
So far so good.
Now, here is an example of a Stroop task in which the prime stimulus is movie violence. Check out the results:
Of course, there is some other weirdness. We get overall mean reaction times for aggressive and non-aggressive words in Table 2. Those could not possibly be right if the means and standard deviations are correct in Table 3. Again, something is just not adding up. I could nitpick a bit and ask why there are no decimal points for the means in each cell, but there are decimal points for standard deviations. That is a very, very minor complaint. The larger complaint is that the authors are not going to be able to say that violent movies increase accessibility of aggressive thoughts based on what they presented using a Stroop task, and that there is a huge (I mean HUGE) disconnect between the overall mean reaction times in Table 2 and the cell means provided in Table 3. After that, all we get are mean difference scores. I don't necessarily have a problem with that, although in Table 5, there is no way that the cell means provided are going to square with the marginal means for violent and non-violent movie conditions. That is also problematic. And if those difference scores are to be believed, it would appear that short-term exposure to a violent movie might actually suppress accessibility of aggressive thoughts among the subsample measuring high in trait aggressiveness, which is not what the authors wish to argue.Again, that is highly problematic. I suppose that the authors made some typos when they were writing up the paper? There is no way of knowing without access to the data set.
Also, Table 4 is just odd. It is unclear just how a MANCOVA would be appropriate as the only DV that the authors consider for the remaining analyses is a difference score. MANOVA and MANCOVA are appropriate analytic techniques for situations in which multiple DVs are analyzed simultaneously. The authors fail to list a covariate. Maybe it is gender? Hard to say. Without an adequate explanation, we as readers are left to guess. Even if a MANCOVA were appropriate, Table 4 is a case study in how not to set up a MANCOVA table. Authors should be explicit about what they are doing as possible. I can read Method and Results sections just fine, thank you. I cannot, however, read minds. Zhang et al. (2013) have placed us all in the position of being mind-readers.
Again, this is a strange article in which the analyses don't make much sense. Something is off in the way the descriptive statistics are reported. Something is off in the way that the inferential statistics are reported. The findings perhaps "looked" superficially good enough to get through peer review. It is possible that depending on the set of reviewers, the findings were ones that fit their own pre-existing beliefs - in other words some form of confirmation bias. I can only speculate on that, and will simply leave it at that. Speculation is not ideal of course.
This particular journal is not exactly a premier journal, which is hardly a knock on it. A number of low-impact journals do no worse, as near as I can tell, then the more premier journals when it comes to the quality of peer review. What matters is that this is an example of an article that slipped through peer review that arguably should not have. Given that this same lab has produced a number of subsequent articles with many of the same problems in more premier journals, I am justifiably worried.
Consider this a cautionary tale.
Reference:
Zhang, Q. , Zhang, D. & Wang, L. (2013). Is Aggressive Trait Responsible for Violence? Priming Effects of Aggressive Words and Violent Movies. Psychology, 4, 96-100. doi: 10.4236/psych.2013.42013
Sunday, May 5, 2019
Some more oddness (Zhang, Xiong, Tian, 2013)
Here's another relevant oldie but goodie by Zhang's lab:
What is odd about this one - aside from Table 3, which is just a mess - is that there is no way that the simple effects F for Boys would be statistically significant given the degrees of freedom. Recall that there are 74 participants in the experiment. That chews up one df. Now let's look at the df for each comparison that would have to be computed, regardless of whether or not it was reported. There are three main effects and a DV which we'll simply note is the difference in reaction time between aggressive and non-aggressive words in a Stroop task. Main effect for movie type costs 1 df, main effect for gender costs another df, and main effect for trait aggressiveness as defined in the article costs 2 df. There are the interactions of movie type x gender (1 df), movie type x trait aggressiveness (2 df) and gender x trait aggressiveness (2 df). The three-way interaction will take another 2 df. That leaves us with 62 df. A significant F-test for simple main effects would need to be practically 4.00. The obtained F in Table 4 for Boys is well below that threshold. And yet the authors claim statistical significance for that particular subsample. The authors want to claim a significant movie type x gender interaction, but have no significant simple main effects. That does not compute, folks.
This, by the way, is one of the better articles from this lab in terms of - at least on the surface - data reporting.
One more thing to note: the authors improperly use terminology commonly used by aggression researchers incorrectly. They are not measuring aggressive attitude levels. They are measuring accessibility of aggressive cognition (or thoughts, in lay terms). I see that mistake enough times to acknowledge that it is hardly unique to Zhang's lab. It is just an annoyance, I suppose.
The article can be read here.
File under articles that probably should not be cited without adequate clarification from the authors.
Reference:
Zhang, Q. , Xiong, D. and Tian, J. (2013) Impact of media violence on aggressive attitude for adolescents. Health, 5, 2156-2161. doi: 10.4236/health.2013.512294
What is odd about this one - aside from Table 3, which is just a mess - is that there is no way that the simple effects F for Boys would be statistically significant given the degrees of freedom. Recall that there are 74 participants in the experiment. That chews up one df. Now let's look at the df for each comparison that would have to be computed, regardless of whether or not it was reported. There are three main effects and a DV which we'll simply note is the difference in reaction time between aggressive and non-aggressive words in a Stroop task. Main effect for movie type costs 1 df, main effect for gender costs another df, and main effect for trait aggressiveness as defined in the article costs 2 df. There are the interactions of movie type x gender (1 df), movie type x trait aggressiveness (2 df) and gender x trait aggressiveness (2 df). The three-way interaction will take another 2 df. That leaves us with 62 df. A significant F-test for simple main effects would need to be practically 4.00. The obtained F in Table 4 for Boys is well below that threshold. And yet the authors claim statistical significance for that particular subsample. The authors want to claim a significant movie type x gender interaction, but have no significant simple main effects. That does not compute, folks.
This, by the way, is one of the better articles from this lab in terms of - at least on the surface - data reporting.
One more thing to note: the authors improperly use terminology commonly used by aggression researchers incorrectly. They are not measuring aggressive attitude levels. They are measuring accessibility of aggressive cognition (or thoughts, in lay terms). I see that mistake enough times to acknowledge that it is hardly unique to Zhang's lab. It is just an annoyance, I suppose.
The article can be read here.
File under articles that probably should not be cited without adequate clarification from the authors.
Reference:
Zhang, Q. , Xiong, D. and Tian, J. (2013) Impact of media violence on aggressive attitude for adolescents. Health, 5, 2156-2161. doi: 10.4236/health.2013.512294
Saturday, May 4, 2019
Imagine (a methods thought experiment)
Imagine a university in an emerging economic power becomes a major research institution early in the decade. Not surprisingly, some researchers within that university quickly take advantage of their university's new status to further their own careers. Fair enough. Who can blame them.
Now let's imagine that one of those researchers publishes work in your area of expertise. It's a large enough university that it would hardly be a surprise. Let's go further. Imagine that you first encounter an article by said researcher. The article seems riddled with multiple errors. You get concerned, as you should, and write the author. In your email, you ask for original data and analyses. You want to be able to reproduce what this researcher claims to have found. You never get a response. You move on with your life.
A couple years later, you realize that said researcher has published multiple articles that have a similar pattern of errors. Sometimes you see identical analyses all the way down to the exact test statistics. We're talking errors in basic descriptive statistics (means and standard deviations, marginal means) as well as inferential statistics (e.g., t-tests and F-tests). These are blatant errors. They are so obvious that you cannot miss them if you do more than a cursory scan of the abstract and discussion section of each article. How are you feeling now? What do you do? Those are questions I struggle with a lot these days. I am not sure my approach was necessarily great - that is to go public - but I do know that I could not just sit still.
This is a real scenario. I personally think that a lot of people should be pissed. I think that the citizens of the country in question should be pissed. After all, their taxes paid for an extended international excursion where some of this dodgy research was presumably worked on. Their tax dollars are funding grants for said author's research even now. I think coauthors should be pissed - especially if the researcher in question did not share data with coauthors. In particular I would advise those coauthors to read about Diederik Stapel and how he went about pulling the proverbial wool over a lot of people's eyes until he eventually got caught. I don't know if this particular situation is in the same league as Stapel, nor will I make that claim (as I have no evidence to do so), but I would not be able to rule that out absent actual data sets to look at (as those would be the evidence needed to rule out foul play). If I were a fellow scientist, I would be pissed, as this author's methodology and analyses are so poorly reported that the body of this author's work could be used to potentially delegitimize a whole area of research - all it takes is a few well-informed lay people to figure out that something is not quite adding up. We have enough problems in the psychological sciences without gross incompetence and potential fraud to worry about.
If you wonder about a few of my seeming obsessions as of the last several months, you at least understand where I am coming from. I am very concerned about the legitimacy of my field first and foremost. Whether or not the author in question publishes research that is in accordance with my professional opinions is not material. For the record, in at least one case, the author was initially preaching to the proverbial choir. In other cases, the author in question had a much harder sell. I am an open-minded skeptic who can be persuaded by good arguments based on solid data and analyses. So that just goes with the territory. We are supposed to be truth seekers and truth tellers who follow the evidence wherever it may lead.
So for now, I will continue to blog about periodic articles from this particular researcher and this particular researcher's lab. Heck, in one way or another I have been discussing this particular problematic research for around three years. Why stop now? This last year I have been much more public. As of yet I see no reason to stop. Neither this blog nor my Twitter account get much in the way of traffic. I am hardly someone who would be taken "seriously" by those who are at the top of the hierarchy in my field, and in my area of expertise. I am, after all, "just an educator" at a regional university no one has heard of. I do think the truth matters, and I am hopeful that the truth will come out.
As the late journalist John Ross used to say, "la lucha sigue, y sigue, y sigue..." This struggle to get the truth out is one that will not stop. Nor will I.
Now let's imagine that one of those researchers publishes work in your area of expertise. It's a large enough university that it would hardly be a surprise. Let's go further. Imagine that you first encounter an article by said researcher. The article seems riddled with multiple errors. You get concerned, as you should, and write the author. In your email, you ask for original data and analyses. You want to be able to reproduce what this researcher claims to have found. You never get a response. You move on with your life.
A couple years later, you realize that said researcher has published multiple articles that have a similar pattern of errors. Sometimes you see identical analyses all the way down to the exact test statistics. We're talking errors in basic descriptive statistics (means and standard deviations, marginal means) as well as inferential statistics (e.g., t-tests and F-tests). These are blatant errors. They are so obvious that you cannot miss them if you do more than a cursory scan of the abstract and discussion section of each article. How are you feeling now? What do you do? Those are questions I struggle with a lot these days. I am not sure my approach was necessarily great - that is to go public - but I do know that I could not just sit still.
This is a real scenario. I personally think that a lot of people should be pissed. I think that the citizens of the country in question should be pissed. After all, their taxes paid for an extended international excursion where some of this dodgy research was presumably worked on. Their tax dollars are funding grants for said author's research even now. I think coauthors should be pissed - especially if the researcher in question did not share data with coauthors. In particular I would advise those coauthors to read about Diederik Stapel and how he went about pulling the proverbial wool over a lot of people's eyes until he eventually got caught. I don't know if this particular situation is in the same league as Stapel, nor will I make that claim (as I have no evidence to do so), but I would not be able to rule that out absent actual data sets to look at (as those would be the evidence needed to rule out foul play). If I were a fellow scientist, I would be pissed, as this author's methodology and analyses are so poorly reported that the body of this author's work could be used to potentially delegitimize a whole area of research - all it takes is a few well-informed lay people to figure out that something is not quite adding up. We have enough problems in the psychological sciences without gross incompetence and potential fraud to worry about.
If you wonder about a few of my seeming obsessions as of the last several months, you at least understand where I am coming from. I am very concerned about the legitimacy of my field first and foremost. Whether or not the author in question publishes research that is in accordance with my professional opinions is not material. For the record, in at least one case, the author was initially preaching to the proverbial choir. In other cases, the author in question had a much harder sell. I am an open-minded skeptic who can be persuaded by good arguments based on solid data and analyses. So that just goes with the territory. We are supposed to be truth seekers and truth tellers who follow the evidence wherever it may lead.
So for now, I will continue to blog about periodic articles from this particular researcher and this particular researcher's lab. Heck, in one way or another I have been discussing this particular problematic research for around three years. Why stop now? This last year I have been much more public. As of yet I see no reason to stop. Neither this blog nor my Twitter account get much in the way of traffic. I am hardly someone who would be taken "seriously" by those who are at the top of the hierarchy in my field, and in my area of expertise. I am, after all, "just an educator" at a regional university no one has heard of. I do think the truth matters, and I am hopeful that the truth will come out.
As the late journalist John Ross used to say, "la lucha sigue, y sigue, y sigue..." This struggle to get the truth out is one that will not stop. Nor will I.
Friday, May 3, 2019
A thread on authoritarianism
Above is the beginning. Worth reading the thread in its entirety. It serves as a good reminder that there is an important person-situation interaction that must be taken into account. In this case, we have to examine the sorts of issues our society's elites are constantly going on about, as their actions (as individuals, corporations, political parties, etc.) can either contain or exacerbate the authoritarian inclinations among that subset of the population that...well...has authoritarian inclinations. We live at a time in which arguably the language used by major media organs (e.g. Fox News), a major political party (GOP), and a White House occupant fan the flames of fear and loathing that pushes those who were already inclined toward authoritarianism over the edge. We've seen this play out before elsewhere historically, as well as in the current moment. I study authoritarianism primarily from the perspective as a social-personality psychologist. I only study from one angle. As Dr. Federico reminds us, the reality is considerably more nuanced.It's taken me a while to read this and formulate some comments, but ultimately my reaction is as negative as most of the others I've seen [thread]. (1/n) https://t.co/qXV9hwNMeY— Christopher Federico (@ChrisPolPsych) May 3, 2019
Wednesday, May 1, 2019
Correcting the scientific record does take a toll on the people involved
May Day was once an international labor day. In the US we tend to avoid so much as a mere acknowledgement of this date and its meaning. It is one worth remembering.
In that spirit, I would like to offer a recent blog post by James Heathers. Part of the labor involved in the sciences includes cleaning up the messes that fellow scientists leave behind when they are grossly incompetent or commit outright fraud. That labor is rarely recognized, is often demonized, and is often done after hours in relative isolation. I don't see that as a sustainable way for individual scientists to live, nor is this a sustainable model for the sciences if we wish them to remain legitimate.
This is as good a time as any to question the incentive structure that allows poor quality work and fraudulent work to slip through the cracks. This is as good a time as any to challenge the publish or perish ethos that has come to define institutions that are not even traditionally research-oriented. The problems revealed by the replication crisis in my field are international in scope and require international solutions. We need to have funding in place for those who are doing the necessary cleanup work as well as structures in place that reward openness - especially when inevitable mistakes get made. We are so far away from where we need to be. We should not see people who are genuinely trying to do the right thing burn out. They need collective support, ASAP.
A wiser man than me once said something about "having nothing to lose but your chains." I think there may be something to that - especially for those of us who toil in relative obscurity and with relative job insecurity.
In that spirit, I would like to offer a recent blog post by James Heathers. Part of the labor involved in the sciences includes cleaning up the messes that fellow scientists leave behind when they are grossly incompetent or commit outright fraud. That labor is rarely recognized, is often demonized, and is often done after hours in relative isolation. I don't see that as a sustainable way for individual scientists to live, nor is this a sustainable model for the sciences if we wish them to remain legitimate.
This is as good a time as any to question the incentive structure that allows poor quality work and fraudulent work to slip through the cracks. This is as good a time as any to challenge the publish or perish ethos that has come to define institutions that are not even traditionally research-oriented. The problems revealed by the replication crisis in my field are international in scope and require international solutions. We need to have funding in place for those who are doing the necessary cleanup work as well as structures in place that reward openness - especially when inevitable mistakes get made. We are so far away from where we need to be. We should not see people who are genuinely trying to do the right thing burn out. They need collective support, ASAP.
A wiser man than me once said something about "having nothing to lose but your chains." I think there may be something to that - especially for those of us who toil in relative obscurity and with relative job insecurity.
Subscribe to:
Posts (Atom)