Greg Francis & Publication Bias Detection
Get flash to fully experience Pearltrees
SPSP 2012: Watchdogs, Witch-hunts, and What to do about False-Positive Findings
In a recent trend, the field of social-personality psychology has become sensitive to the data reporting and analytic strategies that go into the publication of a research paper. Today at the “False Positive Findings are Frequent Findable and Fixable” symposium at SPSP the three speakers presented some very polarizing observations about this trend in our field. The Summary In the first talk, Leslie John of Harvard University bravely discussed the prevalence of questionable data analysis practices in our field. The short answer: People engage in many data collection and analysis strategies that bias hypothesis testing and contribute to the publishing of null findings. In the second talk, Joe Simmons from UPenn presented findings from a research paper suggesting that false findings are preventable with a few key changes in the way people report results and journals review papers.
Rich People May Not Be So Unethical
There was quite the stir a few weeks back about a psychology paper claiming that rich people aren't very nice: Higher social class predicts increased unethical behavior . The article, in PNAS , reported that upper class individuals were more likely to lie, cheat, and break traffic laws. However, these results have been branded "unbelievable" in a Letter to PNAS just published .
Psych your mind has an interesting blog post on using p curves to detect dodgy stats in a a volume of published work (e.g., for a researcher or journal). The idea apparently comes from Uri Simonsohn (one of the authors of a recent paper on dodgy stats ). The author (Michael W. Kraus) bravely plotted and published his own p curve - which looks reasonably 'healthy'. However, he makes an interesting point - which is that we don't know how useful these curves are in practice - which depends among other things on the variability inherent in the profile of p values.
Simulating p curves and detecting dodgy stats
Psychological Statistics: p curves revisited
I finally found some time to take a closer look at p curves. I haven't had a chance to follow-up my simulations (and probably won't for a few weeks if not months), but I have had time to think through the ideas the p curve approach raises based on some of the comments I've received and a brief exchange with Uri Simonsohn (who has answered a few of my questions). First, I got a couple of things at least partly wrong.