background preloader

Peer Review and Integrity in Science

Facebook Twitter

Replication Crisis

EPA Transparency Rule - Secret Science Series. EAT Lancet Paper. Scientific American Goes Full Anti-Science. Back at the beginning of the Trump administration in January 2017, it was all the rage for media on the left to accuse Trump and his people of being “anti-science.”

Scientific American Goes Full Anti-Science

I compiled a collection of such accusations in a post on January 27 of that year, using the title “Who Again Is ‘Anti-Science’?” Among those I cited as making the accusation was the venerable magazine Scientific American, which had published a piece on January 18, 2017 with the title “Trump’s 5 Most Anti-Science Moves.” The four sins of science - and how to overcome them. Share The fact that behavioural economics sits on a throne of lies has not yet reached many policymakers Don't blame journalists for overhyping science - it's often the fault of scientists themselves We need to view scientific advice with a lot more scepticism - especially nutrition and psychology That many scientific fields are rife with error, bias and outright fraud may be unsurprising given the succession of news stories about prominent psychologists faking results and pop-psychology gurus fooling gullible Conservative politicians into ‘power posing’ at party conferences.

The four sins of science - and how to overcome them

But Stuart Ritchie’s Science Fictions: Exposing Fraud, Bias, Negligence and Hype in Science, published this month, demonstrates that similar problems exist throughout virtually all the sciences, including ‘hard’ sciences such as chemistry or medicine that can end up with deadly consequences. No raw data, no science: another possible source of the reproducibility crisis. From Molecular Brain, Biomedical Central Molecular Brain volume 13, Article number: 24 (2020) Cite this article Abstract A reproducibility crisis is a situation where many scientific studies cannot be reproduced.

No raw data, no science: another possible source of the reproducibility crisis

Rein in the four horsemen of irreproducibility. More than four decades into my scientific career, I find myself an outlier among academics of similar age and seniority: I strongly identify with the movement to make the practice of science more robust.

Rein in the four horsemen of irreproducibility

It’s not that my contemporaries are unconcerned about doing science well; it’s just that many of them don’t seem to recognize that there are serious problems with current practices. By contrast, I think that, in two decades, we will look back on the past 60 years — particularly in biomedical science — and marvel at how much time and money has been wasted on flawed research. How can that be? Against Scientism—A Rejoinder to Bo and Ben Winegard. In ancient Athens, shortly after the death of Socrates, word got out that Plato had come up with a definition of man.

Against Scientism—A Rejoinder to Bo and Ben Winegard

Man, according to Plato, was “a featherless biped.” Once he heard this, a philosopher by the name of Diogenes plucked the feathers from a fowl, brought it to Plato’s Academy, and declared, “Behold Plato’s man!” Plato’s definition, as Diogenes’s antics proved, had failed. In their essay “In Defence of Scientism,” Bo and Ben Winegard’s definition of scientism suffers from a similar lack of precision. Scientism, they insist, is simply “the view that scientific attitudes and methods can enhance all modes of empirical inquiry.” Scientism Confuses Method for Metaphysics Although the Winegards present an innocuous definition in their essay, they commonly drift into the less benign form of scientism identified by Hayek. Trouble at the lab - Unreliable research. “I SEE a train wreck looming,” warned Daniel Kahneman, an eminent psychologist, in an open letter last year.

Trouble at the lab - Unreliable research

The premonition concerned research on a phenomenon known as “priming”. Priming studies suggest that decisions can be influenced by apparently irrelevant actions or events that took place just before the cusp of choice. They have been a boom area in psychology over the past decade, and some of their insights have already made it out of the lab and into the toolkits of policy wonks keen on “nudging” the populace. Dr Kahneman and a growing number of his colleagues fear that a lot of this priming research is poorly founded. Motivated Reasoning Is Disfiguring Social Science. On February 15, the American Psychological Association (APA) Council of Representatives voted for a resolution opposing parental spanking (full disclosure: I serve on the APA Council of Representatives but speak only for myself).

Motivated Reasoning Is Disfiguring Social Science

Widely Used Neuroimaging Analyses Allow Almost Any Result To Be Presented As A Successful Replication, Paper Claims. By Matthew Warren As the list of failed replications continues to build, psychology’s reproducibility crisis is becoming harder to ignore.

Widely Used Neuroimaging Analyses Allow Almost Any Result To Be Presented As A Successful Replication, Paper Claims

Now, in a new paper that seems likely to ruffle a few feathers, researchers suggest that even many apparent successful replications in neuroimaging research could be standing on shaky ground. As the paper’s title bluntly puts it, the way imaging results are currently analysed “allows presenting anything as a replicated finding.” The provocative argument is put forward by YongWook Hong from Sungkyunkwan University in South Korea and colleagues, in a preprint posted recently to bioRxiv.

The fundamental problem, say the researchers, is that scientists conducting neuroimaging research tend to make and test hypotheses with reference to large brain structures. This means that strikingly different patterns of brain activity could produce what appears to be the same result. The Mismeasurements of Stephen Jay Gould. Peer Review and Golden Chopsticks.

Academic Activists Send a Published Paper Down the Memory Hole. In the highly controversial area of human intelligence, the ‘Greater Male Variability Hypothesis’ (GMVH) asserts that there are more idiots and more geniuses among men than among women.

Academic Activists Send a Published Paper Down the Memory Hole

Darwin’s research on evolution in the nineteenth century found that, although there are many exceptions for specific traits and species, there is generally more variability in males than in females of the same species throughout the animal kingdom. Evidence for this hypothesis is fairly robust and has been reported in species ranging from adders and sockeye salmon to wasps and orangutans, as well as humans. Multiple studies have found that boys and men are over-represented at both the high and low ends of the distributions in categories ranging from birth weight and brain structures and 60-meter dash times to reading and mathematics test scores. So far, so good. How fake peer review happens: An impersonated reviewer speaks – Retraction Watch. In peer review we (don't) trust: How peer review's filtering poses a systemic risk to science. Cite as: Harry Crane and Ryan Martin (2018).

In peer review we (don't) trust: How peer review's filtering poses a systemic risk to science

In peer review we (don't) trust: How peer review's filtering poses a systemic risk to science. RESEARCHERS.ONE, Abstract: This article describes how the filtering role played by peer review may actually be harmful rather than helpful to the quality of the scientific literature. House of Cards: Is something wrong with the state of science? - Science in the News. Science produces knowledge about our world, while also forming the basis for technologies, medicines and applications that are instrumental to tackling the many problems facing our society. Because of its enormous impact, it is critical that science be built on firm foundations. One of the most crucial differences distinguishing science from pseudoscience (such as many forms of alternative medicine) or non-scientific forms of evidence (such as anecdotes) is reproducibility – the idea that different individuals following the same protocol, or perhaps even using different but comparable approaches, should be able to arrive at the same conclusion.

Fake Publishers Are Ruining Science. Evidence-Based Lies – Blake Gossard. A fundamental tenet of science is that findings must be reproduced. One experiment does not establish new truths. The results have to be replicated by others using the methods described by the original investigators. Replication is key to ensuring that conclusions aren’t spurious. Nevertheless, science is currently plagued by hordes of irreproducible study results.

Researchers: Never Let the Press Office Quote You. The Scientific Method: Hate, Spite, Spleen. In the trillion-dollar global warming controversy, how objective is the science community? Curve-Fitting. Coyote Blog » Blog Archive » Incredible Evidence of P Hacking in Research Studies, Demonstrated with One Chart. Hat tip to Kevin Drum for finding this: I will let him explain the chart, it is worth understanding: The authors collected every significant clinical study of drugs and dietary supplements for the treatment or prevention of cardiovascular disease between 1974 and 2012. Then they displayed them on a scatterplot.Prior to 2000, researchers could do just about anything they wanted. All they had to do was run the study, collect the data, and then look to see if they could pull something positive out of it.

SCIENCE Magazine: Sloppy Reporting. Guest Essay by Kip Hansen I once ran a web site titled the Bad Science Times — I wrote about bad and poor science and science journalism. The Lifespan of a Lie – Trust Issues. How a study about Chronic Fatigue Syndrome was doctored, adding to pain and stigma. The public relies on scientists to report their findings accurately and completely, but that does not always happen. EPA’s Lack of Transparency Is a Breeding Ground for Junk Science.