Computational models that mimic the structure and performance of the human auditory system could help researchers design higher hearing aids, cochlear implants, and brain-machine interfaces. A latest study from MIT has found that modern computational models derived from machine learning are moving closer to this goal.

In the most important study yet of deep neural networks which have been trained to perform auditory tasks, the MIT team showed that almost all of those models generate internal representations that share properties of representations seen within the human brain when individuals are listening to the identical sounds.

The study also offers insight into how you can best train such a model: The researchers found that models trained on auditory input including background noise more closely mimic the activation patterns of the human auditory cortex.

“What sets this study apart is it’s essentially the most comprehensive comparison of those sorts of models to the auditory system to date. The study suggests that models which can be derived from machine learning are a step in the appropriate direction, and it gives us some clues as to what tends to make them higher models of the brain,” says Josh McDermott, an associate professor of brain and cognitive sciences at MIT, a member of MIT’s McGovern Institute for Brain Research and Center for Brains, Minds, and Machines, and the senior writer of the study.

MIT graduate student Greta Tuckute and Jenelle Feather PhD ’22 are the lead authors of the open-access paper, which appears today in .

Models of hearing

Deep neural networks are computational models that consists of many layers of information-processing units that might be trained on huge volumes of information to perform specific tasks. Any such model has turn into widely utilized in many applications, and neuroscientists have begun to explore the likelihood that these systems will also be used to explain how the human brain performs certain tasks.

“These models which can be built with machine learning are capable of mediate behaviors on a scale that basically wasn’t possible with previous varieties of models, and that has led to interest in whether or not the representations within the models might capture things which can be happening within the brain,” Tuckute says.

When a neural network is performing a task, its processing units generate activation patterns in response to every audio input it receives, similar to a word or other form of sound. Those model representations of the input might be in comparison with the activation patterns seen in fMRI brain scans of individuals listening to the identical input.

In 2018, McDermott and then-graduate student Alexander Kell reported that once they trained a neural network to perform auditory tasks (similar to recognizing words from an audio signal), the inner representations generated by the model showed similarity to those seen in fMRI scans of individuals listening to the identical sounds.

Since then, most of these models have turn into widely used, so McDermott’s research group set out to guage a bigger set of models, to see if the power to approximate the neural representations seen within the human brain is a general trait of those models.

For this study, the researchers analyzed nine publicly available deep neural network models that had been trained to perform auditory tasks, and in addition they created 14 models of their very own, based on two different architectures. Most of those models were trained to perform a single task — recognizing words, identifying the speaker, recognizing environmental sounds, and identifying musical genre — while two of them were trained to perform multiple tasks.

When the researchers presented these models with natural sounds that had been used as stimuli in human fMRI experiments, they found that the inner model representations tended to exhibit similarity with those generated by the human brain. The models whose representations were most much like those seen within the brain were models that had been trained on a couple of task and had been trained on auditory input that included background noise.

“For those who train models in noise, they provide higher brain predictions than when you don’t, which is intuitively reasonable because quite a lot of real-world hearing involves hearing in noise, and that’s plausibly something the auditory system is tailored to,” Feather says.

Hierarchical processing

The brand new study also supports the concept the human auditory cortex has a point of hierarchical organization, wherein processing is split into stages that support distinct computational functions. As within the 2018 study, the researchers found that representations generated in earlier stages of the model most closely resemble those seen in the first auditory cortex, while representations generated in later model stages more closely resemble those generated in brain regions beyond the first cortex.

Moreover, the researchers found that models that had been trained on different tasks were higher at replicating different facets of audition. For instance, models trained on a speech-related task more closely resembled speech-selective areas.

“Though the model has seen the very same training data and the architecture is identical, while you optimize for one particular task, you possibly can see that it selectively explains specific tuning properties within the brain,” Tuckute says.

McDermott’s lab now plans to utilize their findings to attempt to develop models which can be much more successful at reproducing human brain responses. Along with helping scientists learn more about how the brain could also be organized, such models may be used to assist develop higher hearing aids, cochlear implants, and brain-machine interfaces.

“A goal of our field is to find yourself with a pc model that may predict brain responses and behavior. We predict that if we’re successful in reaching that goal, it’s going to open quite a lot of doors,” McDermott says.

The research was funded by the National Institutes of Health, an Amazon Fellowship from the Science Hub, an International Doctoral Fellowship from the American Association of University Women, an MIT Friends of McGovern Institute Fellowship, a fellowship from the K. Lisa Yang Integrative Computational Neuroscience (ICoN) Center at MITand a Department of Energy Computational Science Graduate Fellowship.