Uncovering Shoddy Science

By Lida Tunesi

Konrad Kording, professor in the Department of Bioengineering, and colleagues have a new technique for identifying fraudulent scientific papers by spotting reused images. Rather than scrap a failed study, for example, a researcher might attempt to pass off images from a different experiment to give the false impression that their own was a success.

Kording, a Penn Integrates Knowledge (PIK) Professor who also has an appointment in the Department of Neuroscience in Penn’s Perelman School of Medicine, and his collaborators developed an algorithm that can compare images across journal articles and detect such replicas, even if the image has been resized, rotated, or cropped.

They describe their technique in a paper recently published on the BioRxiv preprint server.

“Any fraudulent paper damages science,” Kording says. “In biology, many times fraud is detected when someone looks at a few papers and says ‘hey, these images look a little similar.’ We reckoned we could make an algorithm that does the same thing.”

“Science depends on building upon other people’s work,” adds Daniel Acuna, lead author on the paper, and a student in Kording’s lab at Northwestern University at the time the study was conducted. “If you cannot trust other people’s work, the scientific process collapses and, worse, the general public loses trust in us. Some websites were doing this, anonymously, but at a painstakingly slow rate.” Acuna is now an assistant professor in the School of Information Studies at Syracuse University.

While much of Kording’s work focuses on using data science to understand the brain, he is also curious about the process of research itself, or, as he puts it, “the science of science.” One of the Kording lab’s previous projects closely analyzed common methods of neuroscience research, and another turned a mirror on itself, describing how to structure a scientific paper.

This new study, recently covered in Nature, speaks to Kording’s interest in the publishing stage of research. He also used it as an opportunity to delve into image processing, an important facet of neuroscience. Weaving together such different topics is often what Kording does best.

“As a PIK professor, my role is to be a bridge between fields,” Kording says. “I believe in the importance of that; I enjoy pulling all of these different things together.”

Kording says he and his collaborators are now thinking of licensing out the algorithm to academic journals, but first need to consider some ethical questions. While the algorithm could potentially pick out phony results, it could also generate false accusations if an image reuse was simply a mistake.

“We can detect fraud at scale,” Kording says, “but there can be things that look like fraud that are not.”

Kording is also working with colleagues to start a new academic journal covering the junction between data science and brain science. “We want to innovate and improve the way the publishing system works.”