If you happen to rotate a picture of a molecular structure, a human can tell the rotated image remains to be the identical molecule, but a machine-learning model might think it’s a brand new data point. In computer science parlance, the molecule is “symmetric,” meaning the elemental structure of that molecule stays the identical if it undergoes certain transformations, like rotation.
If a drug discovery model doesn’t understand symmetry, it could make inaccurate predictions about molecular properties. But despite some empirical successes, it’s been unclear whether there may be a computationally efficient method to coach a superb model that’s guaranteed to respect symmetry.
A brand new study by MIT researchers answers this query, and shows the primary method for machine learning with symmetry that’s provably efficient by way of each the quantity of computation and data needed.
These results make clear a foundational query, and so they could aid researchers in the event of more powerful machine-learning models which might be designed to handle symmetry. Such models can be useful in quite a lot of applications, from discovering latest materials to identifying astronomical anomalies to unraveling complex climate patterns.
“These symmetries are essential because they’re some sort of knowledge that nature is telling us in regards to the data, and we must always take it into consideration in our machine-learning models. We’ve now shown that it is feasible to do machine-learning with symmetric data in an efficient way,” says Behrooz Tahmasebi, an MIT graduate student and co-lead creator of this study.
He’s joined on the paper by co-lead creator and MIT graduate student Ashkan Soleymani; Stefanie Jegelka, an associate professor of electrical engineering and computer science (EECS) and a member of the Institute for Data, Systems, and Society (IDSS) and the Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior creator Patrick Jaillet, the Dugald C. Jackson Professor of Electrical Engineering and Computer Science and a principal investigator within the Laboratory for Information and Decision Systems (LIDS). The research was recently presented on the International Conference on Machine Learning.
Studying symmetry
Symmetric data appear in lots of domains, especially the natural sciences and physics. A model that recognizes symmetries is capable of discover an object, like a automotive, irrespective of where that object is placed in a picture, for instance.
Unless a machine-learning model is designed to handle symmetry, it could possibly be less accurate and liable to failure when faced with latest symmetric data in real-world situations. On the flip side, models that make the most of symmetry could possibly be faster and require fewer data for training.
But training a model to process symmetric data is not any easy task.
One common approach is named data augmentation, where researchers transform each symmetric data point into multiple data points to assist the model generalize higher to latest data. For example, one could rotate a molecular structure repeatedly to supply latest training data, but when researchers want the model to be guaranteed to respect symmetry, this might be computationally prohibitive.
An alternate approach is to encode symmetry into the model’s architecture. A widely known example of it is a graph neural network (GNN), which inherently handles symmetric data due to the way it is designed.
“Graph neural networks are fast and efficient, and so they care for symmetry quite well, but no person really knows what these models are learning or why they work. Understanding GNNs is a essential motivation of our work, so we began with a theoretical evaluation of what happens when data are symmetric,” Tahmasebi says.
They explored the statistical-computational tradeoff in machine learning with symmetric data. This tradeoff means methods that require fewer data might be more computationally expensive, so researchers need to search out the fitting balance.
Constructing on this theoretical evaluation, the researchers designed an efficient algorithm for machine learning with symmetric data.
Mathematical combos
To do that, they borrowed ideas from algebra to shrink and simplify the issue. Then, they reformulated the issue using ideas from geometry that effectively capture symmetry.
Finally, they combined the algebra and the geometry into an optimization problem that might be solved efficiently, leading to their latest algorithm.
“Many of the theory and applications were specializing in either algebra or geometry. Here we just combined them,” Tahmasebi says.
The algorithm requires fewer data samples for training than classical approaches, which might improve a model’s accuracy and skill to adapt to latest applications.
By proving that scientists can develop efficient algorithms for machine learning with symmetry, and demonstrating how it could actually be done, these results could lead on to the event of latest neural network architectures that could possibly be more accurate and fewer resource-intensive than current models.
Scientists could also use this evaluation as a start line to look at the inner workings of GNNs, and the way their operations differ from the algorithm the MIT researchers developed.
“Once we all know that higher, we will design more interpretable, more robust, and more efficient neural network architectures,” adds Soleymani.
This research is funded, partly, by the National Research Foundation of Singapore, DSO National Laboratories of Singapore, the U.S. Office of Naval Research, the U.S. National Science Foundation, and an Alexander von Humboldt Professorship.
