While we're on the topic of fraud...

...maybe read this too. In fact, definitely read this, too. So much gets missed in the peer review process, and a fraudster really doesn't need to be especially brilliant to get findings based on dodgy or non-existent data into the body of published findings. I'm a much better peer reviewer, but mostly because I use some recently developed tools of the trade in post-peer review. That said, I use those same post-peer review tools whenever I read much of anything anymore. And yet I know those can only do so much. We're dealing with an entrenched culture that makes it difficult to actually change the behavior. In the meantime, I have seen a series of errata and a corrigendum from a particular lab that are just as troubling as the original papers - in some cases more troubling. And there seems like there is so little to be done about it. That is one thing keeping me up at night.

One postscript - Joe mentions one particular paper where the data were indeed troubling but in which it took the dogged effort of Pat Markey (and later Malte Elson) to get access to the data to try to reproduce it. A retraction followed. Like Joe, I didn't catch it either. Nor would I in the peer review process had I served as a peer reviewer. 

Another Stapel situation?

It's hard to say right now. What is clear is that another PI has seen a couple of his papers retracted due to data irregularities. His former students, post-docs, and coauthors are doing the right thing. At the end of the day, that's what matters. One thing to keep in mind: when we start looking at cases of potential fraud in scientific research, the people most affected are early career researchers (ECRs): grad students and post-docs in particular. With fewer lines on a CV, any retraction will have disproportionate repercussions any time they are on the job market, applying for tenure, etc. All of us, though are negatively affected, to the extent that any article based upon fraudulent (or fabricated) data, to the extent that policy,health, and other personal or professional decisions are based upon that research. The old Russian saying "trust but verify" is seeming more apt all the time. Maybe I'd leave the "trust" part out and just say "verify."

Post-script to the preceding

At the end of the day we as scientists need to discourage efforts to hijack our efforts to address legitimate problems in some of our scientific fields in the name of promoting pseudoscience or engaging in denial of empirically reproduced, replicated, and verified evidence of specific phenomena. It has been a long time since I had any exposure to the work "Merchants of Doubt", which documented the tobacco industry's decades-long (and for a while successful) effort to cast doubt on what was increasingly overwhelming evidence of negative health outcomes associated with using tobacco products. There are merchants of doubt now targeting other phenomena - climate change, immunizations for childhood diseases, etc. We as scientists have scarce resources. The less legitimacy we give to these contemporary merchants of doubt, in the form of conference attendance, membership dues, etc., the better. For me, that is easy. I only get partially funded for one conference per fiscal year. I have to make that conference count for something.

When a conference on the crisis in confidence is not what it appears

Read Dorothy Bishop's blog on the matter here. Not every organization that appears on the surface to show interest in improving the replicability and reproducibility of scientific results is actually interested in doing so. Anything sponsored by the National Association of Scholars (which apparently uses the same acronym as the genuinely respected National Academy of Sciences) is best treated with some suspicion. The choice of acronym alone seems designed to confuse. Beyond that, Dr. Bishop does a good job of exposing this organization's political agenda. In other words, this bunch is interested in improving science as much as it is protecting a particular political perspective in line with the current occupant in the White House with regard to climate science.

Happy New Year!

I wish to thank each of you who has visited over the last few years and to wish each of you a wonderful 2020.

My plans for this blog in 2020 are simple enough. I still have some unfinished business with research from a very error-prone lab, I want to continue to examine allegiance effects in weapons effect research, and since this is an election year, I expect I will spend a bit of time offering a few pointers on media literacy. The last bit will likely be a bit of an update of some posts from late 2016 as I was processing the aftermath of the US Presidential election. Expect to see posts advocating for reform in my corner of the sciences. Expect new links - in particular ones to resources for those who value post-peer review as much as I do.

In the meantime, let's enjoy a moment to celebrate and to enjoy those things that truly matter. On second thought, let's be generous with the time that we spend enjoying those things that truly matter.

Onward.

Data sleuthing made easy

You all know that I have done just a bit of data sleuthing here or there. I do so with no real fancy background in statistics. I have sufficient course work to teach stats courses at the undergraduate level, but I am no quantitative psychologist. So, I appreciate articles like How to Be a Statistical Detective. The author lays out some common problems and how any of us can use our already existing skills to detect those problems. I use some of these resources already, and am reasonably adept at using a calculator. I will likely add more links to these resources to this blog.

This article is behind a paywall, but I suspect my more enterprising readers already know how to obtain a copy. This article is fundamental reading.

Oblique Strategies

I found the card deck of Oblique Strategies developed by Brian Eno and Peter Schmidt (1975) to be quite useful. Although I have never owned an original deck (those were quite pricey back when I was a grad student), thanks to the early days of the world wide web, I could find early sites that would generate Oblique Strategies that I would use in my creative process while working on my dissertation. Now, well, into the 21st century, Daniel Lakens has developed this cool shiny app that will provide random Oblique Strategies for your own inspiration. I will make this link available elsewhere on my blog so that it is easily accessible.

What drew me to the dilemmas posed by the deck - or virtual deck - was that they required a certain amount of willingness to think "outside the box" or outside of one's normal professional parameters. From my vantage point, there is something healthy about that. Give it a try. See how your writing changes. See how you view everything from study design to data analytic strategies, to - yes - writing up a research report. At the end of the day we social scientists are still creators. We may be data driven creators, but creators nonetheless. Besides, we should have some fun with our work as truth seekers and truth tellers.