Wwhen Sholto David quit his job last fall, he could have looked for another job, taken time to travel, or grabbed his tent and hopped on his bike. But David, a biologist living in Pontypridd, Wales, threw his efforts into a somewhat obscure hobby: finding errors in scientific papers and doing his best to correct them.
The job, says David, is largely thankless. Academics often became defensive about their studies or refused to respond to his criticisms. Magazine editors took a similar tack, ignoring his letters, dismissing them, or investigating them on timescales bordering on the glacial.
But then come an announcement. Last week, one of the most prestigious cancer centers in the US, the Harvard-affiliated Dana-Farber Cancer Institute in Boston, said it was seeking to retract six research papers and correct 31 more after David raised concerns in a blog about dozens of his studies. Many were led by top executives at the institute.
Of the rest of the papers David flagged, Dr Barrett Rollins, the integrity research officer at Dana-Farber, said one remains under investigation and three require no further action. Sixteen contain data generated in other laboratories and where possible the heads of those laboratories were contacted. “We will work with them to see that they correct the literature as warranted,” he said.
The move came as a “big surprise”, David said. “To the credit of the Dana-Farber Cancer Institute, I am of course happy that they are going to do the corrections and retractions. But at the same time it does leave a sour taste in the mouth. Most of the time it just doesn’t happen. People ignore you, institutions insist it takes years to do investigations, and journals drag their feet at every opportunity.”
According to Rollins, the institute, following usual practices of “reviewing any potential data error and making corrections when warranted,” has already taken “swift and decisive action” in 97% of the cases David flagged where his scientists were primary authors. Rollins himself is an author of some of the papers flagged and has recused himself from any relevant investigations.
David, who did a PhD in molecular biology at Newcastle University, has long been fascinated by flaws in science. He began to find flaws in systematic reviews, meta-analyses and clinical trials, which he would show to the responsible scientists and the journals that published them. When those efforts were fruitless, he turned to posting PubPeera website where scientists can comment on published papers.
It became more than a pastime. The biologist has flagged around 2,000 papers on PubPeer, most due to concerns about possible image manipulation. At first, he identified cases by eye, looking for duplications and questionable manipulations that stretched images, cut out specific features, or joined parts together. Now he has help from Imagetwin, AI-based software that compares images to a database of more than 25 million photos that have appeared in open-access journals.
“A lot of it is sloppiness and my thought is that people have higher expectations,” David said. “When people think about science, when they donate to science and cancer campaigns, the expectation is that the scientists doing this research have high standards and are very careful in what they do.”
The wave of retractions and corrections follows a big project in 2021 who found that researchers could reproduce results from only half of the top pre-clinical cancer studies they looked at. In many cases, when the experiments were repeated, any positive outcomes were much smaller than originally claimed. “Perhaps part of it is because the work being done is not of high quality,” said David. “It ultimately all boils down to a lack of repeatability.”
David stopped short of alleging wrongdoing and emphasized that he did not want to create an environment where scientists felt harassed or afraid to publish. But equally, he said, repeated errors were troubling.
“How many errors is acceptable before we think something more alarming is happening?” he said. “If you comb through a lot of people’s papers you will find mistakes, but at some point there has to be a limit to how many sloppy mistakes you make before it’s something else, that it’s not something you can call an honest mistake don’t dismiss .”
Rollins said: “The presence of image anomalies in a newspaper is not evidence of an author’s intent to deceive. That conclusion can only be reached after a careful, fact-based investigation, which is an integral part of our answer. Our experience is that mistakes are often unintentional and do not rise to the level of misconduct.”
As for David, he thinks he will return to more conventional work in a year or two. “I’m just spending some savings,” he said. “Maybe I’ll travel a bit this year. I sound good, but I’m not. I’m just stretching my meager savings and it’s cheap rent out here.”
