Biology is a wondrous yet delicate tapestry. At the center is DNA, the master weaver that encodes proteins, accountable for orchestrating the various biological functions that sustain life throughout the human body. Nonetheless, our body is akin to a finely tuned instrument, liable to losing its harmony. In spite of everything, we’re faced with an ever-changing and relentless natural world: pathogens, viruses, diseases, and cancer. 

Imagine if we could expedite the means of creating vaccines or drugs for newly emerged pathogens. What if we had gene editing technology able to routinely producing proteins to rectify DNA errors that cause cancer? The hunt to discover proteins that may strongly bind to targets or speed up chemical reactions is significant for drug development, diagnostics, and diverse industrial applications, yet it is usually a protracted and expensive endeavor.

To advance our capabilities in protein engineering, MIT CSAIL researchers got here up with “FrameDiff,” a computational tool for creating latest protein structures beyond what nature has produced. The machine learning approach generates “frames” that align with the inherent properties of protein structures, enabling it to construct novel proteins independently of preexisting designs, facilitating unprecedented protein structures.

“In nature, protein design is a slow-burning process that takes hundreds of thousands of years. Our technique goals to supply a solution to tackling human-made problems that evolve much faster than nature’s pace,” says MIT CSAIL PhD student Jason Yim, a lead creator on a latest paper concerning the work. “The aim, with respect to this latest capability of generating synthetic protein structures, opens up a myriad of enhanced capabilities, resembling higher binders. This implies engineering proteins that may attach to other molecules more efficiently and selectively, with widespread implications related to targeted drug delivery and biotechnology, where it could end in the event of higher biosensors. It could even have implications for the sphere of biomedicine and beyond, offering possibilities resembling developing more efficient photosynthesis proteins, creating more practical antibodies, and engineering nanoparticles for gene therapy.” 

Framing FrameDiff

Proteins have complex structures, made up of many atoms connected by chemical bonds. Crucial atoms that determine the protein’s 3D shape are called the “backbone,” sort of just like the spine of the protein. Every triplet of atoms along the backbone shares the identical pattern of bonds and atom types. Researchers noticed this pattern might be exploited to construct machine learning algorithms using ideas from differential geometry and probability. That is where the frames are available in: Mathematically, these triplets might be modeled as rigid bodies called “frames” (common in physics) which have a position and rotation in 3D. 

These frames equip each triplet with enough information to find out about its spatial surroundings. The duty is then for a machine learning algorithm to learn the way to move each frame to construct a protein backbone. By learning to construct existing proteins, the algorithm hopefully will generalize and find a way to create latest proteins never seen before in nature.

Training a model to construct proteins via “diffusion” involves injecting noise that randomly moves all of the frames and blurs what the unique protein looked like. The algorithm’s job is to maneuver and rotate each frame until it looks like the unique protein. Though easy, the event of diffusion on frames requires techniques in stochastic calculus on Riemannian manifolds. On the idea side, the researchers developed “SE(3) diffusion” for learning probability distributions that nontrivially connects the translations and rotations components of every frame.

The subtle art of diffusion

In 2021, DeepMind introduced AlphaFold2, a deep learning algorithm for predicting 3D protein structures from their sequences. When creating synthetic proteins, there are two essential steps: generation and prediction. Generation means the creation of recent protein structures and sequences, while “prediction” means determining what the 3D structure of a sequence is. It’s no coincidence that AlphaFold2 also used frames to model proteins. SE(3) diffusion and FrameDiff were inspired to take the concept of frames further by incorporating frames into diffusion models, a generative AI technique that has grow to be immensely popular in image generation, like Midjourney, for instance. 

The shared frames and principles between protein structure generation and prediction meant the most effective models from each ends were compatible. In collaboration with the Institute for Protein Design on the University of Washington, SE(3) diffusion is already getting used to create and experimentally validate novel proteins. Specifically, they combined SE(3) diffusion with RosettaFold2, a protein structure prediction tool very like AlphaFold2, which led to “RFdiffusion.” This latest tool brought protein designers closer to solving crucial problems in biotechnology, including the event of highly specific protein binders for accelerated vaccine design, engineering of symmetric proteins for gene delivery, and robust motif scaffolding for precise enzyme design. 

Future endeavors for FrameDiff involve improving generality to problems that mix multiple requirements for biologics resembling drugs. One other extension is to generalize the models to all biological modalities including DNA and small molecules. The team posits that by expanding FrameDiff’s training on more substantial data and enhancing its optimization process, it could generate foundational structures boasting design capabilities on par with RFdiffusion, all while preserving the inherent simplicity of FrameDiff. 

“Discarding a pretrained structure prediction model [in FrameDiff] opens up possibilities for rapidly generating structures extending to large lengths,” says Harvard University computational biologist Sergey Ovchinnikov. The researchers’ modern approach offers a promising step toward overcoming the constraints of current structure prediction models. Despite the fact that it’s still preliminary work, it’s an encouraging stride in the best direction. As such, the vision of protein design, playing a pivotal role in addressing humanity’s most pressing challenges, seems increasingly within sight, because of the pioneering work of this MIT research team.” 

Yim wrote the paper alongside Columbia University postdoc Brian Trippe, French National Center for Scientific Research in Paris’ Center for Science of Data researcher Valentin De Bortoli, Cambridge University postdoc Emile Mathieu, and Oxford University professor of statistics and senior research scientist at DeepMind Arnaud Doucet. MIT professors Regina Barzilay and Tommi Jaakkola advised the research. 

The team’s work was supported, partially, by the MIT Abdul Latif Jameel Clinic for Machine Learning in Health, EPSRC grants and a Prosperity Partnership between Microsoft Research and Cambridge University, the National Science Foundation Graduate Research Fellowship Program, NSF Expeditions grant, Machine Learning for Pharmaceutical Discovery and Synthesis consortium, the DTRA Discovery of Medical Countermeasures Against Latest and Emerging threats program, the DARPA Accelerated Molecular Discovery program, and the Sanofi Computational Antibody Design grant. This research will probably be presented on the International Conference on Machine Learning in July.