False positives: fraud and misconduct are threatening scientific research. Dirk Smeesters had spent several years of his career as a social psychologist at Erasmus University in Rotterdam studying how consumers behaved in different situations.
Did colour have an effect on what they bought? How did death-related stories in the media affect how people picked products? From Walter Stewart to Uri Simonsohn « The Hardest Science. Over on G+, Ole Rogeberg asks what ever happened to Walter Stewart?
Stewart was a biologist employed by NIH in the 80s and 90s who became involved in rooting out questionable research practices. Rogeberg posts an old Omni Magazine interview with Stewart (from 1989) in which Stewart describes how he got involved in investigating fraud and misconduct and what led him to think that it was more widespread than many scientists were willing to acknowledge. If you have been following the fraud scandals in psychology and the work of Uri Simonsohn, you should read it. It is completely riveting. And I found some of the parallels to be uncanny.
Replication studies: Bad copy. For many psychologists, the clearest sign that their field was in trouble came, ironically, from a study about premonition.
Daryl Bem, a social psychologist at Cornell University in Ithaca, New York, showed student volunteers 48 words and then abruptly asked them to write down as many as they could remember. Next came a practice session: students were given a random subset of the test words and were asked to type them out. Bem found that some students were more likely to remember words in the test if they had later practised them.
Effect preceded cause. Bem published his findings in the Journal of Personality and Social Psychology (JPSP) along with eight other experiments1 providing evidence for what he refers to as “psi”, or psychic effects. Positive results in psychology can behave like rumours: easy to release but hard to dispel. Uncertainty shrouds psychologist's resignation. Uri Simonsohn, the researcher who flagged up questionable data in studies by social psychologist Dirk Smeesters, has revealed the name of a second social psychologist whose data he believes to be suspiciously perfect.
That researcher is Lawrence Sanna, whose former employer, the University of Michigan in Ann Arbor, tells Simonsohn that he resigned his professorship there at the end of May. The reasons for Sanna's resignation are not known, but it followed questions from Simonsohn and a review by Sanna’s previous institution, the University of North Carolina in Chapel Hill (UNC). According to the editor of the Journal of Experimental Social Psychology, Sanna has also asked that three of his papers be retracted from the journal. In both Smeesters’ and Sanna’s work, odd statistical patterns in the data raised concerns with Simonsohn, at the University of Pennsylvania in Philadelphia.
But the similarity between the cases ends there. University withdraws articles - News - Erasmus Universiteit Rotterdam. Erasmus University Rotterdam has decided to withdraw two articles by Professor Smeesters, Professor of Consumer and Society at Rotterdam School of Management, Erasmus University from scientific journals.
The University’s action follows a report from the Inquiry Committee on Scientific Integrity, set up after suspicions arose that the professor had committed scientific errors. The Committee investigated how a large number of publications were established. Two articles were found to have irregularities with findings that, in a statistical sense, are highly unlikely. The raw data forming the basis of these articles was not available for inspection by third parties, and the professor indicated that he had selected data so that the sought-after effects were statistically significant.
The titles of these two articles are:- Johnson, C.S., Smeesters, D.H.R.V. & Wheeler, S.C. (2012). This action is in violation of the principles of academic integrity. Following investigation, Erasmus social psychology professor retracts two studies, resigns. Mysterious Whistleblower Identified in New Social Psychology Scandal. Skip to main content Science/AAAS Subscribe Main menu You are here News » ScienceInsider » People & Events » Mysterious Whistleblower Identified in New Social Psychology Scandal Martin Enserink Share on facebookShare on twitterShare on redditMore Sharing ServicesShare on email ScienceInsider Mysterious Whistleblower Identified in New Social Psychology Scandal 28 June 2012 10:24 am 5 Comments.
Lord of the Files, part 2 - Counterbalanced Blog. Back in November, I wrote about the Diederik Stapel affair that was rocking the psychological community.
Now, it’s all happening again – recently, Uri Simonsohn has found evidence suggesting that another Dutch social psychologist, Dirk Smeesters, has been tinkering with data to produce more desirable outcomes in his research. We’re still waiting on Simonsohn’s paper on the matter to be published, so I think it’s a little too soon to speculate on how he came across Smeesters’ work – some have already started to make melodramatic comments likening Simonsohn’s approach to medieval torture.
Clearly, we need to be very cautious about using such approaches as a sledgehammer. We don’t want to end up in a situation whereby completely innocent researchers wrongly getting caught out. But there are also some tough questions that need to be asked about how and why this sort of behaviour is happening. The data detective. Luke Church Photography Psychology was already under scrutiny following a series of high-profile controversies.
Now it faces fresh questions over research practices that can sometimes produce eye-catching — but irreproducible — results. An interview with Uri Simonsohn, the data sleuth behind the Smeesters psychology misconduct case. Simonsohn Responds (Translated to English) Navel gazing on steroids in social and personality science. It seems a critical mass of papers, blogs, articles, and comments has hit the field of social and personality psychology in the last few months.
So much so, that I thought it would be constructive to catalogue everything that has emerged from Lehrer, to Bem, to Stapel, to Bargh, to Simonsohn, and on to Francis (borrowing freely from Sanjay Srivastava’s hard work). I would like this post to be a resource of sorts for the whole sordid affair, which can be used for methods courses taught in the not too distant future.
So, if I’ve missed anything important please send the link or paper my way. The denouement: Lehrer & Bem The story probably starts with Lehrer’s piece on the decline effect, which was followed closely by the news of Bem’s article on the existence of ESP. Just recently, Ritchie et al (2012) published their failure to replicate Bem’s research. Jumping the shark: Stapel. SPSP Groups. Fraud-Detection Tool Could Shake Up Psychology. AMSTERDAM—The most startling thing about the latest scandal to hit social psychology isn’t the alleged violation of scientific ethics itself, scientists say, or the fact that it happened in the Netherlands, the home of fallen research star and serial fraudster Diederik Stapel, whose case shook the field to its core less than a year ago.
Is post-hoc statistical analysis the new fraud detection tool? A new review looks at fraudster Reuben’s work.