The source of the trouble is “paper mills,” businesses or individuals that charge fees to publish fake studies in legitimate journals under the names of desperate scientists whose careers depend on their publishing record.
“The entire structure of science could collapse if this is left unaddressed,” said study author Luís Amaral, a physicist at Northwestern University.
Paper mills look for weak links, such as lax verification protocols, in the typically rigorous publication machinery, then exploit those to place hundreds of fabricated studies with vulnerable journals or publishers, according to scientist investigators who have been tracking and cataloging their work.
It can be a costly mess to clean up.
Publishers who have become aware of suspected paper mill activity have been forced to retract hundreds of papers at once, and in some cases shut down journals.
After retracting more than 11,300 papers that appeared compromised, Wiley, a U.S.-based publisher with a portfolio of 1,600 journals, last year announced it would shutter 19 of its publications, including some that had shown signs of takeover by paper mills.
More recently, Taylor & Francis, an academic publisher that produces more than 2,700 journals, announced in July it would stop taking submissions to its journal Bioengineered while its editors investigated past papers for paper mill activity.
But tracking the scale of these organized operations across the body of scientific literature has been difficult. When paper mills are detected, they change their tactics, and few publishers disclose when they have been targeted.
“We only know about Wiley because they have been transparent about the way that they were trying to handle these issues,” said Reese Richardson, a data scientist who studies publishing at Northwestern University who is an author on the PNAS study.
The authors of the study created a database of suspected paper mill papers, over 32,700 in all, from nearly every publisher, by assembling the work of other volunteer investigators who have been cataloging groups of studies with similar patterns that appear to come from paper mills.
With a big-data approach that spans the published scientific record, the new study confirms trends that scientist investigators of shoddy papers have shown in case studies for years.
The new study “harvests data from multiple places and triangulates signals from multiple sources to provide one of the first landscape views of the problem,” said Guillaume Cabanac, a professor of computer science at the University of Toulouse in France, one of the architects of a tool that detects faulty work in papers.
Cabanac, who wasn’t involved in the PNAS study, said that more scientists and publishers are aware of the problem than a few years ago, when paper mills were viewed as a fringe issue.
The growing role of AI in all business sectors makes the trend that the new paper documents especially concerning, said James Evans, a sociologist who studies science and technology at the University of Chicago, who wasn’t involved with the study.
Because large language models are consuming scientific literature without discriminating between legitimate papers and fraudulent ones, paper mills “have the potential to really muddy the waters of science and scientific understanding,” Evans said.
In an analysis of articles in the journal PLOS One, the authors demonstrate how paper mills could proliferate by targeting editors at a journal. They found that between 2006 and 2023, more than 700 articles were retracted, out of nearly 277,000 published. Of more than 18,300 editors who worked on the published articles, 22 had an unusually high rate of retraction. While they handled 0.2% of published articles, they refereed 19% of retracted studies, Richardson said.
The authors declined to name the editors they identified, but said the findings suggested that each was in charge of far more problematic papers than could be explained by chance.
Renee Hoch, head of publication ethics at PLOS, said in an email that the publisher has been aware of paper mills manipulating peer review and has in the past removed compromised papers and editors when they have been found.
She added that new AI and manual tools have helped reject submissions before faulty papers were published.
Some publishers who have investigated suspected paper mill activity at their own journals have acknowledged that people presenting themselves as editors, or hijacking the identity of genuine researchers, is a strategy that paper mills use. Some have instituted more strict ID checks for editors as a result.
In another analysis in the PNAS paper, the authors tracked duplicated images—photographs of sections of tissue, microscope images of cells or DNA analyses in gels appear in identical form in multiple papers. Much like fingerprints, such biological photographs are expected to be unique, so identical images from separate experiments have come to be recognized as an indicator of a mistake or potential fraud.
The authors analyzed reports of duplicated images submitted on PubPeer, a forum where researchers post about problematic papers. They found reports of 2,213 papers containing 4,188 instances where two articles shared the same images, a level of replication that suggested that some of the images had been misappropriated.
While publishers have increasingly acknowledged these operations and said they have taken steps to spot fraud, the authors of the new study show that retractions—in which journals pull papers that appear to have flawed or faulty data—aren’t keeping pace with the growth of paper mills.
In 2020, 2,905 published articles were retracted, but more than 4,500 suspected paper mill products were logged in their database. Given that the study’s estimates of paper mill activity are likely an undercount, Richardson said, “We know that this barely scratches the surface of what’s out there.”
Write to Nidhi Subbaraman at nidhi.subbaraman@wsj.com
