From cameras to self-driving cars, lots of today’s technologies depend upon artificial intelligence to extract meaning from visual information. Today’s AI technology has artificial neural networks at its core, and more often than not we will trust these AI computer vision systems to see things the way in which we do — but sometimes they falter. In keeping with MIT and IBM research scientists, one option to improve computer vision is to instruct the unreal neural networks that they depend on to deliberately mimic the way in which the brain’s biological neural network processes visual images.

Researchers led by MIT Professor James DiCarlo, the director of MIT’s Quest for Intelligence and member of the MIT-IBM Watson AI Lab, have made a pc vision model more robust by training it to work like a component of the brain that humans and other primates depend on for object recognition. This May, on the International Conference on Learning Representations, the team reported that once they trained a synthetic neural network using neural activity patterns within the brain’s inferior temporal (IT) cortex, the unreal neural network was more robustly capable of discover objects in images than a model that lacked that neural training. And the model’s interpretations of images more closely matched what humans saw, even when images included minor distortions that made the duty tougher.

Comparing neural circuits

A lot of the unreal neural networks used for computer vision already resemble the multilayered brain circuits that process visual information in humans and other primates. Just like the brain, they use neuron-like units that work together to process information. As they’re trained for a selected task, these layered components collectively and progressively process the visual information to finish the duty — determining, for instance, that a picture depicts a bear or a automotive or a tree.

DiCarlo and others previously found that when such deep-learning computer vision systems establish efficient ways to unravel visual problems, they find yourself with artificial circuits that work similarly to the neural circuits that process visual information in our own brains. That’s, they grow to be surprisingly good scientific models of the neural mechanisms underlying primate and human vision.

That resemblance helps neuroscientists deepen their understanding of the brain. By demonstrating ways visual information will be processed to make sense of images, computational models suggest hypotheses about how the brain might accomplish the identical task. As developers proceed to refine computer vision models, neuroscientists have found recent ideas to explore in their very own work.

“As vision systems recover at performing in the true world, a few of them grow to be more human-like of their internal processing. That’s useful from an understanding-biology viewpoint,” says DiCarlo, who can be a professor of brain and cognitive sciences and an investigator on the McGovern Institute for Brain Research.

Engineering a more brain-like AI

While their potential is promising, computer vision systems are usually not yet perfect models of human vision. DiCarlo suspected one option to improve computer vision could also be to include specific brain-like features into these models.

To check this concept, he and his collaborators built a pc vision model using neural data previously collected from vision-processing neurons within the monkey IT cortex — a key a part of the primate ventral visual pathway involved in the popularity of objects — while the animals viewed various images. More specifically, Joel Dapello, a Harvard University graduate student and former MIT-IBM Watson AI Lab intern; and Kohitij Kar, assistant professor and Canada Research Chair (Visual Neuroscience) at York University and visiting scientist at MIT; in collaboration with David Cox, IBM Research’s vp for AI models and IBM director of the MIT-IBM Watson AI Lab; and other researchers at IBM Research and MIT asked a synthetic neural network to emulate the behavior of those primate vision-processing neurons while the network learned to discover objects in a regular computer vision task.

“In effect, we said to the network, ‘please solve this standard computer vision task, but please also make the function of certainly one of your inside simulated “neural” layers be as similar as possible to the function of the corresponding biological neural layer,’” DiCarlo explains. “We asked it to do each of those things as best it could.” This forced the unreal neural circuits to seek out a distinct option to process visual information than the usual, computer vision approach, he says.

After training the unreal model with biological data, DiCarlo’s team compared its activity to a similarly-sized neural network model trained without neural data, using the usual approach for computer vision. They found that the brand new, biologically informed model IT layer was — as instructed — a greater match for IT neural data.  That’s, for each image tested, the population of artificial IT neurons within the model responded more similarly to the corresponding population of biological IT neurons.

The researchers also found that the model IT was also a greater match to IT neural data collected from one other monkey, despite the fact that the model had never seen data from that animal, and even when that comparison was evaluated on that monkey’s IT responses to recent images. This indicated that the team’s recent, “neurally aligned” computer model could also be an improved model of the neurobiological function of the primate IT cortex — an interesting finding, on condition that it was previously unknown whether the quantity of neural data that will be currently collected from the primate visual system is able to directly guiding model development.

With their recent computer model in hand, the team asked whether the “IT neural alignment” procedure also results in any changes in the general behavioral performance of the model. Indeed, they found that the neurally-aligned model was more human-like in its behavior — it tended to reach appropriately categorizing objects in images for which humans also succeed, and it tended to fail when humans also fail.

Adversarial attacks

The team also found that the neurally aligned model was more proof against “adversarial attacks” that developers use to check computer vision and AI systems. In computer vision, adversarial attacks introduce small distortions into images that are supposed to mislead a synthetic neural network.

“Say that you might have a picture that the model identifies as a cat. Because you might have the knowledge of the inner workings of the model, you may then design very small changes within the image in order that the model suddenly thinks it’s not a cat,” DiCarlo explains.

These minor distortions don’t typically idiot humans, but computer vision models struggle with these alterations. A one that looks on the subtly distorted cat still reliably and robustly reports that it’s a cat. But standard computer vision models usually tend to mistake the cat for a dog, or perhaps a tree.

“There have to be some internal differences in the way in which our brains process images that result in our vision being more proof against those sorts of attacks,” DiCarlo says. And indeed, the team found that once they made their model more neurally aligned, it became more robust, appropriately identifying more images within the face of adversarial attacks. The model could still be fooled by stronger “attacks,” but so can people, DiCarlo says. His team is now exploring the boundaries of adversarial robustness in humans.

Just a few years ago, DiCarlo’s team found they may also improve a model’s resistance to adversarial attacks by designing the primary layer of the unreal network to emulate the early visual processing layer within the brain. One key next step is to mix such approaches — making recent models which can be concurrently neurally aligned at multiple visual processing layers.

The brand new work is further evidence that an exchange of ideas between neuroscience and computer science can drive progress in each fields. “Everybody gets something out of the exciting virtuous cycle between natural/biological intelligence and artificial intelligence,” DiCarlo says. “On this case, computer vision and AI researchers get recent ways to attain robustness, and neuroscientists and cognitive scientists get more accurate mechanistic models of human vision.”

This work was supported by the MIT-IBM Watson AI Lab, Semiconductor Research Corporation, the U.S. Defense Research Projects Agency, the MIT Shoemaker Fellowship, U.S. Office of Naval Research, the Simons Foundation, and Canada Research Chair Program.