Sunday, August 25, 2019
Will this time be different?
I had the pleasure of seeing and hearing Sanjay speak these words live in early July at the closing of the SIPS conference in Rotterdam. I really hope those words are heeded. Simply tightening up some methods without addressing the social inequality that afflicts our science (as is the case with so many sciences) is insufficient. If the only people who benefit are those who just happen to keep paying membership dues, we've failed. Open science is intersectional and is a social movement. Anything short of that will be a failure. I hope that I do not find myself in a decade asking the same question that a good friend of mine once asked over three decades ago in his zine, Pressure: "So, where's the change?"
Thursday, August 1, 2019
Some initial impressions about SIPS 2019
I think perhaps the best way to start is with a Twitter thread I posted right as we were about to end:
My experience started with the preconference put together by the repliCATS project. Their travel grant to those willing to participate in the preconference is what made going to SIPS possible for me. During the 5th and 6th of July, I spent the entire work day at the conference site with a team of several other psychologists in various phases of their careers (most were postdocs and grad students). Each team was tasked with the responsibility of assessing the probability of replication for 25 claims. We had a certain amount of time to read each claim, look up the relevant article, look up any other supplementary materials relevant to the task, and then to make our predictions. We then discussed our initial assessments and recalibrated. I found the process engaging and enlightening. What was cool was how each of us brought some unique expertise to the table. Four of the claims we assessed were meta-analytic, and since I was the one person in the room who had conducted published meta-analyses, I became the de facto expert on that methodology. Believe me when I say that I still don't feel like an expert. But okay. So for those claims, my peers quizzed me a good deal and I got to share a few things that might be useful about raw effect sizes, assessment of publication bias, etc. We did the same with others. There were some claims that all of us found vexing. Goes with the territory. I think what I got from the experience was how much we in the psychological sciences really need each other if we are going to move the field forward. I also realized just how talented the current generation of early career researchers truly is. As deflating as some of the research we evaluated was, I could not help but feel a certain level of cautious optimism about the future of the psychological sciences.
The conference itself started mid-day on the 7th of July and lasted through the 9th of July. It was filled with all sorts of sessions - hackathons, workshops, and unconferences. I tended to gravitate to the latter two classes of sessions during the conference this time around. The conference format itself is rather loose. One could start a session, decide "no this is not for me" and leave without worrying about hurt feelings. One could walk in to a session in the middle and be welcomed. It was great to go to a professional meeting without seeing one person wearing business attire. At least that level of pretense was dispensed with, for which I am grateful. I gravitated toward sessions on the last day that had a specific focus on inclusiveness. That is a topic that has been on my mind going back to my student activist days in the 1980s. There is a legitimate concern that by the time all is said and done, we'll manage to fix some methodological problems that are genuinely troubling for the discipline without addressing the problem that there are a lot of talented people who could offer so much who are shut out due to their ethnicity, national origin (especially if from the Global South), sexual orientation, and gender identity. If all we get as the same power structure with somewhat better methods, we will have failed, in my personal and professional opinion. I think the people leading at least a couple of those sessions seemed to get that. I hope that those who are running SIPS get it too. Maybe an inclusiveness hackathon is in order? I guess I am volunteering myself to lead that one just by blogging about it!
This was also an interesting conference in that I am both mid-career and primarily an educator. So, I was definitely part of a small subset of attendees. Personally, I felt pretty engaged. I can see how one in similar circumstances might end up feeling legitimately left out. There was some effort to have sessions devoted to mid-career researchers. We may want that expanded to mid-career educators as well. We too may want to be active participants in creating a better science of psychology, but our primary means of doing so is going to be in the classroom and not via published reports. Those of us who may one day become part of administration (dept chairs, deans, etc.) or who already occupy those position should have some forum to discuss how we can better educate the next generation of Psychology majors at the undergraduate level so that they are both better consumers of research and better prepared for the changes occurring that will impact them as they enter graduate programs (for that subset of majors who will do so).
It was also great to finally meet up in person with a number of people with whom I have interacted via Twitter DM, email, and sometimes via phone. That experience was beautiful. There is something about actually getting to interact in person that is truly irreplaceable.
Given the attendance at the conference, I can see how much of a logistical challenge the organizers faced. There were moments where last minute room changes did not quite get communicated. The dinner was one where participants were mostly underfed (that is probably more on the restaurant than the organizers, and I can chalk that up to "stuff happens" and leave it at that). Maintaining that sense of intimacy with a much larger than anticipated group was a challenge. But I never felt isolated or alone. There was always a session of interest. There was always someone to talk to. The sense of organized chaos is one that should be maintained.
I did find time to wander around the city of Rotterdam, in spite of my relative lack of down time for this conference. The fact that it was barely past the Summer Solstice meant that I got some good daylight quality photos well into the evening. I got to know the "cool district" of Rotterdam quite well, and definitely went off the beaten path in the process. I'll share some of those observations at another time.
Overall, this was a great experience. I will likely participate in the future, as I am finding ways of using what I learned in the classroom. As long as participants can come away from the experience thinking and knowing that there were more good sessions than they could possibly attend, it will continue to succeed. If the organizers are serious about inclusiveness, they will have something truly revolutionary as part of their legacy. Overall, the sum total of this set of experiences left me cautiously optimistic in a way that I have not been in a very long time. There is hope yet for the psychological sciences, and I got to meet some of the people who are providing the reason for that hope. Perhaps I will meet others who did not attend this year at future conferences. I'll hold out hope for that as well.
The conference was different from any conference I have ever experienced. For those wanting to get a feel for what SIPS is about, a good place to begin might be to check out the page for this year's conference. Rotterdam was a good location in part because the city slogan is make it happen. SIPS is an organization devoted to actively changing the way the science of psychology is done, and is formatted in such a way that those participating become active. This is a conference for people who really want to roll up their sleeves and get involved.1. With #SIPS2019 about to come to a close Wednesday, I have a few observations - brief for now. I love the format and have felt more actively engaged in a conference than I have in a very long time.— James Benjamin (@AJBenjaminJr) July 8, 2019
My experience started with the preconference put together by the repliCATS project. Their travel grant to those willing to participate in the preconference is what made going to SIPS possible for me. During the 5th and 6th of July, I spent the entire work day at the conference site with a team of several other psychologists in various phases of their careers (most were postdocs and grad students). Each team was tasked with the responsibility of assessing the probability of replication for 25 claims. We had a certain amount of time to read each claim, look up the relevant article, look up any other supplementary materials relevant to the task, and then to make our predictions. We then discussed our initial assessments and recalibrated. I found the process engaging and enlightening. What was cool was how each of us brought some unique expertise to the table. Four of the claims we assessed were meta-analytic, and since I was the one person in the room who had conducted published meta-analyses, I became the de facto expert on that methodology. Believe me when I say that I still don't feel like an expert. But okay. So for those claims, my peers quizzed me a good deal and I got to share a few things that might be useful about raw effect sizes, assessment of publication bias, etc. We did the same with others. There were some claims that all of us found vexing. Goes with the territory. I think what I got from the experience was how much we in the psychological sciences really need each other if we are going to move the field forward. I also realized just how talented the current generation of early career researchers truly is. As deflating as some of the research we evaluated was, I could not help but feel a certain level of cautious optimism about the future of the psychological sciences.
The conference itself started mid-day on the 7th of July and lasted through the 9th of July. It was filled with all sorts of sessions - hackathons, workshops, and unconferences. I tended to gravitate to the latter two classes of sessions during the conference this time around. The conference format itself is rather loose. One could start a session, decide "no this is not for me" and leave without worrying about hurt feelings. One could walk in to a session in the middle and be welcomed. It was great to go to a professional meeting without seeing one person wearing business attire. At least that level of pretense was dispensed with, for which I am grateful. I gravitated toward sessions on the last day that had a specific focus on inclusiveness. That is a topic that has been on my mind going back to my student activist days in the 1980s. There is a legitimate concern that by the time all is said and done, we'll manage to fix some methodological problems that are genuinely troubling for the discipline without addressing the problem that there are a lot of talented people who could offer so much who are shut out due to their ethnicity, national origin (especially if from the Global South), sexual orientation, and gender identity. If all we get as the same power structure with somewhat better methods, we will have failed, in my personal and professional opinion. I think the people leading at least a couple of those sessions seemed to get that. I hope that those who are running SIPS get it too. Maybe an inclusiveness hackathon is in order? I guess I am volunteering myself to lead that one just by blogging about it!
This was also an interesting conference in that I am both mid-career and primarily an educator. So, I was definitely part of a small subset of attendees. Personally, I felt pretty engaged. I can see how one in similar circumstances might end up feeling legitimately left out. There was some effort to have sessions devoted to mid-career researchers. We may want that expanded to mid-career educators as well. We too may want to be active participants in creating a better science of psychology, but our primary means of doing so is going to be in the classroom and not via published reports. Those of us who may one day become part of administration (dept chairs, deans, etc.) or who already occupy those position should have some forum to discuss how we can better educate the next generation of Psychology majors at the undergraduate level so that they are both better consumers of research and better prepared for the changes occurring that will impact them as they enter graduate programs (for that subset of majors who will do so).
It was also great to finally meet up in person with a number of people with whom I have interacted via Twitter DM, email, and sometimes via phone. That experience was beautiful. There is something about actually getting to interact in person that is truly irreplaceable.
Given the attendance at the conference, I can see how much of a logistical challenge the organizers faced. There were moments where last minute room changes did not quite get communicated. The dinner was one where participants were mostly underfed (that is probably more on the restaurant than the organizers, and I can chalk that up to "stuff happens" and leave it at that). Maintaining that sense of intimacy with a much larger than anticipated group was a challenge. But I never felt isolated or alone. There was always a session of interest. There was always someone to talk to. The sense of organized chaos is one that should be maintained.
I did find time to wander around the city of Rotterdam, in spite of my relative lack of down time for this conference. The fact that it was barely past the Summer Solstice meant that I got some good daylight quality photos well into the evening. I got to know the "cool district" of Rotterdam quite well, and definitely went off the beaten path in the process. I'll share some of those observations at another time.
Overall, this was a great experience. I will likely participate in the future, as I am finding ways of using what I learned in the classroom. As long as participants can come away from the experience thinking and knowing that there were more good sessions than they could possibly attend, it will continue to succeed. If the organizers are serious about inclusiveness, they will have something truly revolutionary as part of their legacy. Overall, the sum total of this set of experiences left me cautiously optimistic in a way that I have not been in a very long time. There is hope yet for the psychological sciences, and I got to meet some of the people who are providing the reason for that hope. Perhaps I will meet others who did not attend this year at future conferences. I'll hold out hope for that as well.
Subscribe to:
Posts (Atom)