background preloader

The Decline Effect and the Scientific Method

The Decline Effect and the Scientific Method
On September 18, 2007, a few dozen neuroscientists, psychiatrists, and drug-company executives gathered in a hotel conference room in Brussels to hear some startling news. It had to do with a class of drugs known as atypical or second-generation antipsychotics, which came on the market in the early nineties. The drugs, sold under brand names such as Abilify, Seroquel, and Zyprexa, had been tested on schizophrenics in several large clinical trials, all of which had demonstrated a dramatic decrease in the subjects’ psychiatric symptoms. As a result, second-generation antipsychotics had become one of the fastest-growing and most profitable pharmaceutical classes. By 2001, Eli Lilly’s Zyprexa was generating more revenue than Prozac. But the data presented at the Brussels meeting made it clear that something strange was happening: the therapeutic power of the drugs appeared to be steadily waning. Before the effectiveness of a drug can be confirmed, it must be tested and tested again.

Nearer, better Absence makes your heart grow fonder, but close quarters may boost your career. According to new research by scientists at Harvard Medical School, the physical proximity of researchers, especially between the first and last author on published papers, strongly correlates with the impact of their work. “Despite all of the profound advances in information technology, such as video conferencing, we found that physical proximity still matters for research productivity and impact,” says Isaac Kohane, the Lawrence J. Henderson Professor of Pediatrics at Children’s Hospital Boston and director of the Countway Library of Medicine at Harvard Medical School. The research was published in the Dec. 15 issue of PLoS ONE. Given that the Internet and social networks enable people to collaborate remotely, the researchers investigated whether proximity corresponded to the scientific impact of research as measured by citations of resulting publications. Digital drive

News Desk: More Thoughts on the Decline Effect In “The Truth Wears Off,” I wanted to explore the human side of the scientific enterprise. My focus was on a troubling phenomenon often referred to as the “decline effect,” which is the tendency of many exciting scientific results to fade over time. This empirical hiccup afflicts fields from pharmacology to evolutionary biology to social psychology. There is no simple explanation for the decline effect, but the article explores several possibilities, from the publication biases of peer-reviewed journals to the “selective reporting” of scientists who sift through data. This week, the magazine published four very thoughtful letters in response to the piece. The first letter, like many of the e-mails, tweets, and comments I’ve received directly, argues that the decline effect is ultimately a minor worry, since “in the long run, science prevails over human bias.” That’s a pretty perfect example of selective reporting in science. But that’s not always the case. Illustration: Laurent Cilluffo

Why was that paper retracted? Editor to Retraction Watch: “It’s none of your damn business” L. Henry Edmunds, photo by University of Pennsylvania Yesterday, we reported on the retraction of a 2004 study in the Annals of Thoracic Surgery. As we noted, the notice’s language was, um, fuzzy, referring vaguely to an investigation by the University of Florida, which uncovered instances of repetitious, tabulated data from previously published studies. Today, we are slightly more clear, although what we really got was an earful of other language. We had the pleasure of speaking this morning with L. It’s none of your damn business. Ranting against “journalists and bloggists,” Edmunds, a cardiac surgeon at the University of Pennsylvania, said the purpose of the retraction notice was merely to inform our reader that the article is retracted. Curiosity and details be damned! If you get divorced from your wife, the public doesn’t need to know the details. Sarah Palin’s coterie couldn’t have said it better. I cured cancer. We’re sure Edmunds wouldn’t accept such a manuscript. Update, 12:30 p.m.

When Peer Review Falters - Room for Debate Mohamed Gad-el-Hak is the Inez Caudill Eminent Professor of Mechanical & Nuclear Engineering at Virginia Commonwealth University. The peer review system in all fields is not infallible. There are too many submitted manuscripts, and competent reviewers are often overwhelmed. We need to place a lot more emphasis on the quality of research than on quantity. The submission of any author may end up in the hands of an incompetent reviewer who may accept the paper simply because he or she does not know any better. Or it may land in the hands of a competent reviewer who is too busy to give the manuscript a thorough reading, and accepts a mediocre work out of guilty feelings. Editors try to go around this problem by seeking the advice of more than one reviewer. The present system of peer review in many fields is on the verge of breaking down, but it is still the only thing we have.

Do peer reviewers get worse with experience? Plus a poll Peer review isn’t a core subject of this blog. We leave that to the likes of Nature’s Peer-to-Peer, or even the Dilbert Blog. But it seems relevant to look at the peer review process for any clues about how retracted papers are making their way into press. We’re not here to defend peer review against its many critics. We have the same feelings about it that Churchill did about democracy, aka the worst form of government except for all those others that have been tried. Still, peer review is meant as a barrier between low-quality papers and publication, and it often comes up when critics ask questions such as, “How did that paper ever get through peer review?” With that in mind, a paper published last week in the Annals of Emergency Medicine caught our eye. How bad did they get? So the decline was slow. Such a negative overall trend is contrary to most editors’ and reviewers’ intuitive expectations and beliefs about reviewer skills and the benefits of experience. What can be done?

The shroud of retraction: Virology Journal withdraws paper about whether Christ cured a woman with flu It takes decades, and even centuries, to overturn the Catholic canon of law, but medical journals move much more quickly: Just three weeks after the Virology Journal published a paper speculating that a woman described in the Bible as being “cured by our Lord Jesus Christ” had flu, the journal has apologized for ever posting it online. After bemused — to put it mildly — reactions from bloggers Bob O’Hara (who alerted us to the retraction), P.Z. Myers, and Tara C. Smith, as well as questions from a journal reader, the journal’s editor, Robert F. Garry, posted a retraction to O’Hara’s blog, and in his own journal: Presumably, this is not what the editors of Virology Journal mean when they call the journal “a strategic alternative to the traditional virology communication process.” O’Hara’s tongue-in-cheek yet exasperated analysis of the paper is worth the read and deserves the traffic, so I will send you there instead of quote it extensively. Update, 11:30 a.m. Like this: Like Loading...

Research : fopen( [ Notice: Undefined variable: xmlContent in /var/www/rr_public/libraries/rrservice/service.php on line 22 Notice: Trying to get property of non-object in /var/www/rr_public/components/com_news/views/article/tmpl/default.php on line 12 Notice: Trying to get property of non-object in /var/www/rr_public/components/com_news/views/article/tmpl/default.php on line 12 Notice: Trying to get property of non-object in /var/www/rr_public/components/com_news/views/article/tmpl/default.php on line 15 Notice: Trying to get property of non-object in /var/www/rr_public/components/com_news/views/article/tmpl/default.php on line 15 Notice: Trying to get property of non-object in /var/www/rr_public/components/com_news/views/article/tmpl/default.php on line 18 Notice: Trying to get property of non-object in /var/www/rr_public/components/com_news/views/article/tmpl/default.php on line 18

Publishing your science paper is only the half the job | David Dobbs | Science Not all papers achieve museum space like Einstein's General Theory of Relativity. Photograph: David Silverman/Getty Images Perhaps the oddest and least predictable scientific conference I attend is ScienceOnline, a version of which met earlier this month at the British Library. That event, ScienceOnline London, or SOLO, is a spinoff of the original ScienceOnline held every January in the United States. One idea that took a higher profile than I had expected this year was that the scientific establishment has come to wildly overvalue and overemphasise the scientific paper. For starters (the argument goes), the paper offers, in this age of instant, data-rich communication, a horribly slow and expensive way to share data and ideas. Ditching or devaluing the paper poses challenges, of course. Good questions, and they sparked juicy debates at the conference. This struck me during one of the many discussions at SOLO of whether and how scientists should engage the public.

Are Peer-Reviewers Overloaded? Or Are Their Incentives Misaligned When multiple scientists come up with similar solutions simultaneously, it may be time to take notice. In this week’s issue of Science, a letter to the editor, titled “Battling the Paper Glut,” proposed a solution to finding competent reviewers: Journals should demand that for every paper submitted, an author provide three reviews of other manuscripts If it sounds familiar, it is. The “golden rule” to peer-review was reviewed just last week in the Scholarly Kitchen in a fully formulated proposal to privatize the peer review system (see: “Privatizing Peer Review — The PubCred Proposal“) In my review, I wrote that the PubCred bank solution was based on a tentative premise (that editors are experiencing a more difficult time finding competent reviewers), which is based on an even more tentative premise (that competent reviewers are overloaded with requests and others are unwilling to pull their fair share of the reviewing load). Like this: Like Loading...

Nobelist Linda Buck retracts two studies on olfactory networks — and the news is embargoed Well, it’s happened: The Embargo Watch and Retraction Watch worlds have collided. I had initially figured on two posts here, but it soon became clear that how journals were handling these retractions, using embargoes, was central to both. So this is being cross-posted on both blogs. Linda Buck, who shared the 2004 Nobel Prize in Physiology or Medicine, has retracted two papers published in 2005 and 2006. Both retractions — one in the Proceedings of the National Academy of Sciences (PNAS) and one in Science — appear online today. The papers describe how nerves that carry information about scents connect from the nose to the olfactory bulb, where they are processed. The retractions come two and a half years after Buck retracted a 2001 Nature paper co-authored with Zhihua Zou, a post-doc in her then-Harvard lab. The PNAS and Science retractions are of those two later publications. The Science retraction reads: Here’s how the Hutchinson Center responded to questions we sent Buck: In 2008, Dr.