Devorah Fischler
John Crocker had expected to see a flat line — a familiar horizontal track with some slight peaks and valleys — but the plot of energy in front of him dove sharply downward.
“It’s a once-in-a-lifetime finding,” says Crocker. “It was as if the simulation had unexpectedly fallen into a deep canyon on an energy surface. This was lucky for two reasons. Firstly, it turned out to be a game changer for our study of glassy materials. And secondly, similar canyons have the potential to help others grappling with the same computational obstacles we face in our field, from computer scientists working on machine learning algorithms to bioengineers studying protein folding. We ended up with significant results because we were curious enough to try a method that shouldn’t have worked. But it did.”
The method is metadynamics, a computational approach to exploring energy landscapes. Its counterintuitive application is the subject of a recent publication in PNAS from a group of Penn Engineers at the University of Pennsylvania led by Crocker, Professor and Graduate Group Chair in the Department of Chemical and Biomolecular Engineering (CBE), along with Robert Riggleman, Associate Professor in CBE, and Amruthesh Thirumalaiswamy, Ph.D. student in CBE.
Most solids are glasses (or glassy). We categorize the rest as crystals. These categorizations are not limited to glass or crystal as we might imagine them, but instead indicate how atoms in any solid are arranged. Crystals have neat, repetitive atomic structures. Glasses, however, are amorphous. Their atoms and molecules take on a vast number of disordered configurations.
Glassy configurations get stuck while pursuing — as all systems do — their most stable, lowest energy states. Given enough time, glasses will still very slowly relax in energy, but their disordered atoms make it a slow and difficult process.
Low-energy, stable glasses, or “ideal glasses,” are the key to a storehouse of knowledge that researchers are keen to unlock.
Seeking to understand and eventually replicate the conditions of glassy materials that overcome the obstacles of their own atomic quirks, scientists use both experimental and theoretical approaches.
Labs have, for example, melted and re-cooled fossilized amber to develop processes for recreating the encouraging effects that millions of years have had on its glassy pursuit of low-energy states. Crocker’s team, affiliated with the cross-disciplinary Penn Institute for Computational Science (PICS), explores physical structures with mathematical models.
“We use computational models to simulate the positions and movements of atoms in different glasses,” says Thirumalaiswamy. “In order to keep track of a material’s particles, which are so numerous and dynamic they are impossible to visualize in three dimensions, we need to represent them mathematically in high-dimensional virtual spaces. If we have 300 atoms, for example, we need to represent them in 900 dimensions. We call these energy landscapes. We then investigate the landscapes, navigating them almost like explorers.”
In these computational models, single configuration points, digests of atomic movement, tell the story of a glass’ energy levels. They show where a glass has gotten stuck and where it might have achieved a low-energy state.
The problem is that until now, researchers have not been able to navigate landscapes efficiently enough to find these rare instances of stability.
“Most studies do random walks around high-dimensional landscapes at enormous computational cost. It would take an infinite amount of time to find anything of interest. The landscapes are immense, and these walks are repetitive, wasting large amounts of time fixed in a single state before moving on to the next one,” says Riggleman.
And so, they took a chance in trying metadynamics, a method that seemed destined to fail.
Metadynamics is an algorithmic strategy developed to explore the entire landscape and avoid repetition. It assigns a penalty for going back to the same place twice. Metadynamics never works in high-dimensional spaces, however, because it takes too long to construct the penalties, canceling out the strategy’s potential for efficiency.
Yet as the researchers watched their configuration energy trend downward, they realized it had succeeded.
“We couldn’t have guessed it, but the landscapes proved to have these canyons with floors that are only two- or three-dimensional,” says Crocker. “Our algorithm literally fell right in. We found regularly occurring low-energy configurations in several different glasses with a method we think could be revolutionary for other disciplines as well.”
The potential applications of the Crocker Lab canyons are wide-ranging.
In the two decades since the Human Genome Project finished its mapping, scientists have been using computational models to fold peptide sequences into proteins. Proteins that fold well in nature have, through evolution, found ways to explore low-energy states analogous to those of ideal glasses.
Theoretical studies of proteins use energy landscapes to learn about the folding processes that create the functional (or dysfunctional) foundations for biological health. Yet measuring these structures takes time, money and energy that scientists and the populations they aim to serve don’t have to spare. Bogged down by the same computational inefficiencies that glassy materials researchers face, genomic scientists may find similar successes with metadynamics-based approaches, accelerating the pace of medical research.
Machine learning processes have a lot in common with random walks in high-dimensional space. Training artificial intelligence takes an enormous amount of computational time and power and has a long way to go in terms of predictive accuracies.
A neural net needs to “see,” for example, thousands to millions of faces in order to acquire enough skill for facial recognition. With a more strategic computational process, machine learning could become faster, cheaper and more accessible. The metadynamics algorithm may have the potential to overcome the need for the huge and costly datasets typical of the process.
Not only would this provide solutions for industry efficiency, but it could also democratize AI, allowing people with modest resources to do their own training and development.
“We’re conjecturing that the landscapes in these different fields have similar geometric structures to ours,” says Crocker. “We suspect there might be a deep mathematical reason for why these canyons exist, and they may be present in these other related systems. This is our invitation; we look forward to the dialogue it begins.”
This work was supported by NSF-Division of Material Research 1609525 and 1720530 and computational resources provided by XSEDE (Extreme Science and Engineering Discovery Environment) through TG-DMR150034.