To make our way through the world, our brain must develop an intuitive understanding of the physical world around us, which we then use to interpret sensory information coming into the brain.

How does the brain develop that intuitive understanding? Many scientists consider that it could use a process just like what’s referred to as “self-supervised learning.” This kind of machine learning, originally developed as a method to create more efficient models for computer vision, allows computational models to find out about visual scenes based solely on the similarities and differences between them, with no labels or other information.

A pair of studies from researchers on the K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center at MIT offers recent evidence supporting this hypothesis. The researchers found that once they trained models referred to as neural networks using a specific variety of self-supervised learning, the resulting models generated activity patterns very just like those seen within the brains of animals that were performing the identical tasks because the models.

The findings suggest that these models are in a position to learn representations of the physical world that they will use to make accurate predictions about what’s going to occur in that world, and that the mammalian brain could also be using the identical strategy, the researchers say.

“The theme of our work is that AI designed to assist construct higher robots finally ends up also being a framework to higher understand the brain more generally,” says Aran Nayebi, a postdoc within the ICoN Center. “We will’t say if it’s the entire brain yet, but across scales and disparate brain areas, our results appear to be suggestive of an organizing principle.”

Nayebi is the lead writer of certainly one of the studies, co-authored with Rishi Rajalingham, a former MIT postdoc now at Meta Reality Labs, and senior authors Mehrdad Jazayeri, an associate professor of brain and cognitive sciences and a member of the McGovern Institute for Brain Research; and Robert Yang, an assistant professor of brain and cognitive sciences and an associate member of the McGovern Institute. Ila Fiete, director of the ICoN Center, a professor of brain and cognitive sciences, and an associate member of the McGovern Institute, is the senior writer of the opposite study, which was co-led by Mikail Khona, an MIT graduate student, and Rylan Schaeffer, a former senior research associate at MIT.

Each studies can be presented on the 2023 Conference on Neural Information Processing Systems (NeurIPS) in December.

Modeling the physical world

Early models of computer vision mainly relied on supervised learning. Using this approach, models are trained to categorise images which are each labeled with a reputation — cat, automotive, etc. The resulting models work well, but the sort of training requires an ideal deal of human-labeled data.

To create a more efficient alternative, in recent times researchers have turned to models built through a way referred to as contrastive self-supervised learning. This kind of learning allows an algorithm to learn to categorise objects based on how similar they’re to one another, with no external labels provided.

“This can be a very powerful method because you’ll be able to now leverage very large modern data sets, especially videos, and really unlock their potential,” Nayebi says. “Lots of the fashionable AI that you simply see now, especially within the last couple years with ChatGPT and GPT-4, is a result of coaching a self-supervised objective function on a large-scale dataset to acquire a really flexible representation.”

These kind of models, also called neural networks, consist of 1000’s or hundreds of thousands of processing units connected to one another. Each node has connections of various strengths to other nodes within the network. Because the network analyzes huge amounts of knowledge, the strengths of those connections change because the network learns to perform the specified task.

Because the model performs a specific task, the activity patterns of various units throughout the network may be measured. Each unit’s activity may be represented as a firing pattern, just like the firing patterns of neurons within the brain. Previous work from Nayebi and others has shown that self-supervised models of vision generate activity just like that seen within the visual processing system of mammalian brains.

In each of the brand new NeurIPS studies, the researchers got down to explore whether self-supervised computational models of other cognitive functions may additionally show similarities to the mammalian brain. Within the study led by Nayebi, the researchers trained self-supervised models to predict the longer term state of their environment across a whole bunch of 1000’s of naturalistic videos depicting on a regular basis scenarios.    

“For the last decade or so, the dominant method to construct neural network models in cognitive neuroscience is to coach these networks on individual cognitive tasks. But models trained this manner rarely generalize to other tasks,” Yang says. “Here we test whether we are able to construct models for some aspect of cognition by first training on naturalistic data using self-supervised learning, then evaluating in lab settings.”

Once the model was trained, the researchers had it generalize to a task they call “Mental-Pong.” This is analogous to the video game Pong, where a player moves a paddle to hit a ball traveling across the screen. Within the Mental-Pong version, the ball disappears shortly before hitting the paddle, so the player has to estimate its trajectory with a view to hit the ball.

The researchers found that the model was in a position to track the hidden ball’s trajectory with accuracy just like that of neurons within the mammalian brain, which had been shown in a previous study by Rajalingham and Jazayeri to simulate its trajectory — a cognitive phenomenon referred to as “mental simulation.” Moreover, the neural activation patterns seen throughout the model were just like those seen within the brains of animals as they played the sport — specifically, in a component of the brain called the dorsomedial frontal cortex. No other class of computational model has been in a position to match the biological data as closely as this one, the researchers say.

“There are various efforts within the machine learning community to create artificial intelligence,” Jazayeri says. “The relevance of those models to neurobiology hinges on their ability to moreover capture the inner workings of the brain. The undeniable fact that Aran’s model predicts neural data is admittedly vital because it suggests that we could also be getting closer to constructing artificial systems that emulate natural intelligence.”

Navigating the world

The study led by Khona, Schaeffer, and Fiete focused on a variety of specialized neurons referred to as grid cells. These cells, situated within the entorhinal cortex, help animals to navigate, working along with place cells situated within the hippocampus.

While place cells fire each time an animal is in a particular location, grid cells fire only when the animal is at certainly one of the vertices of a triangular lattice. Groups of grid cells create overlapping lattices of various sizes, which allows them to encode a lot of positions using a comparatively small variety of cells.

In recent studies, researchers have trained supervised neural networks to mimic grid cell function by predicting an animal’s next location based on its place to begin and velocity, a task referred to as path integration. Nevertheless, these models hinged on access to privileged details about absolute space in any respect times — information that the animal doesn’t have.                               

Inspired by the striking coding properties of the multiperiodic grid-cell code for space, the MIT team trained a contrastive self-supervised model to each perform this same path integration task and represent space efficiently while doing so. For the training data, they used sequences of velocity inputs. The model learned to tell apart positions based on whether or not they were similar or different — nearby positions generated similar codes, but further positions generated more different codes.    

“It’s just like training models on images, where if two images are each heads of cats, their codes ought to be similar, but when one is the pinnacle of a cat and one is a truck, you then want their codes to repel,” Khona says. “We’re taking that very same idea but applying it to spatial trajectories.”

Once the model was trained, the researchers found that the activation patterns of the nodes throughout the model formed several lattice patterns with different periods, very just like those formed by grid cells within the brain.

“What excites me about this work is that it makes connections between mathematical work on the striking information-theoretic properties of the grid cell code and the computation of path integration,” Fiete says. “While the mathematical work was analytic — what properties does the grid cell code possess? — the approach of optimizing coding efficiency through self-supervised learning and obtaining grid-like tuning is synthetic: It shows what properties is perhaps mandatory and sufficient to clarify why the brain has grid cells.”

The research was funded by the K. Lisa Yang ICoN Center, the National Institutes of Health, the Simons Foundation, the McKnight Foundation, the McGovern Institute, and the Helen Hay Whitney Foundation.