Melissa PappasIn 2005, John Ioannidis published a bombshell paper titled “Why Most Published Research Findings Are False.” In it, Ioannidis argued that a lack of scientific rigor in biomedical research — such as poor study design, small sample sizes and improper assessment of the significance of data— meant that a large percentage of experiments would not return the same results if they were conducted again.
Since then, researchers’ awareness of this “replication crisis” has grown, especially in fields that directly impact the health and wellbeing of people, where lapses in rigor can have life-or-death consequences. Despite this attention and motivation, however, little progress has been made in addressing the roots of the problem. Formal training in rigorous research practices remains rare; while mentors advise their students on how to properly construct and conduct experiments to produce the most reliable evidence, few educational resources exist to support them.
To address this discrepancy, the National Institute of Neurological Disorders and Stroke (NINDS), part of the National Institutes of Health (NIH), has launched the Initiative to Improve Education in the Principles of Rigorous Research.
Konrad Kording, a Penn Integrates Knowledge Professor with appointments in the Departments of Bioengineering and Computer and Information Science in Penn Engineering and the Department of Neuroscience in Penn’s Perelman School of Medicine, has been awarded one of the initiative’s first five grants.
“The replication crisis is real,” says Kording. “I’ve tried to replicate the research of others and failed. I’ve reanalyzed my own data and found major mistakes that needed to be corrected. I was never properly taught how to do rigorous science, and I want to improve that for the next generation.”
Known as “Creating an Educational Nexus for Training in Experimental Rigor,” or CENTER, the grant will support Kording and his colleagues as they develop a user-friendly, open-source educational platform of modules that address biases in research, logical fallacies around causality, hypothesis development, literature search design, identifying experimental variables, and reducing confounding variables in research.
In a time when remote work and digital accessibility are priorities, research training will need to meet the same standards. Kording has already demonstrated leadership in that space: He and his colleagues developed NeuroMatch, a not-for-profit networking and educational online platform for computational neuroscientists, as an answer to maintaining professional connection during the pandemic when in-person conferences were unavailable.
Kording’s work in creating an inclusive space for this platform will provide him with the skills to bring CENTER to reality as an innovative and accessible way to learn from experts in the field of biomedicine. And, the usage and feedback data that is collected in CENTER will inform the modules in real time, allowing both machine learning and student mentoring to occur simultaneously.
CENTER’s educational modules are funded by four additional grants for the Materials to Enhance Training in Experimental Rigor (METER). Kording will guide the recipients of these four awards and evaluate the resources produced. He will also direct the initiative as a whole and host annual meetings of award recipients over the next five years.
“The goal,” Kording says, “is to help the worldwide research community come together around improving rigor in science.”