For nearly a decade, a team of MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers have been in search of to uncover why certain images persist in a people’s minds, while many others fade. To do that, they got down to map the spatio-temporal brain dynamics involved in recognizing a visible image. And now for the primary time, scientists harnessed the combined strengths of magnetoencephalography (MEG), which captures the timing of brain activity, and functional magnetic resonance imaging (fMRI), which identifies energetic brain regions, to exactly determine when and where the brain processes a memorable image.
Their open-access study, published this month in , used 78 pairs of images matched for a similar concept but differing of their memorability scores — one was highly memorable and the opposite was easy to forget. These images were shown to fifteen subjects, with scenes of skateboarding, animals in various environments, on a regular basis objects like cups and chairs, natural landscapes like forests and beaches, urban scenes of streets and buildings, and faces displaying different expressions. What they found was that a more distributed network of brain regions than previously thought are actively involved within the encoding and retention processes that underpin memorability.
“People are likely to remember some images higher than others, even after they are conceptually similar, like different scenes of an individual skateboarding,” says Benjamin Lahner, an MIT PhD student in electrical engineering and computer science, CSAIL affiliate, and first writer of the study. “We have identified a brain signature of visual memorability that emerges around 300 milliseconds after seeing a picture, involving areas across the ventral occipital cortex and temporal cortex, which processes information like color perception and object recognition. This signature indicates that highly memorable images prompt stronger and more sustained brain responses, especially in regions just like the early visual cortex, which we previously underestimated in memory processing.”
While highly memorable images maintain a better and more sustained response for about half a second, the response to less memorable images quickly diminishes. This insight, Lahner elaborated, could redefine our understanding of how memories form and persist. The team envisions this research holding potential for future clinical applications, particularly in early diagnosis and treatment of memory-related disorders.
The MEG/fMRI fusion method, developed within the lab of CSAIL Senior Research Scientist Aude Oliva, adeptly captures the brain’s spatial and temporal dynamics, overcoming the normal constraints of either spatial or temporal specificity. The fusion method had somewhat help from its machine-learning friend, to raised examine and compare the brain’s activity when taking a look at various images. They created a “representational matrix,” which is sort of a detailed chart, showing how similar neural responses are in various brain regions. This chart helped them discover the patterns of where and when the brain processes what we see.
Picking the conceptually similar image pairs with high and low memorability scores was the crucial ingredient to unlocking these insights into memorability. Lahner explained the strategy of aggregating behavioral data to assign memorability scores to pictures, where they curated a various set of high- and low-memorability images with balanced representation across different visual categories.
Despite strides made, the team notes just a few limitations. While this work can discover brain regions showing significant memorability effects, it cannot elucidate the regions’ function in the way it is contributing to raised encoding/retrieval from memory.
“Understanding the neural underpinnings of memorability opens up exciting avenues for clinical advancements, particularly in diagnosing and treating memory-related disorders early on,” says Oliva. “The precise brain signatures we have identified for memorability could lead on to early biomarkers for Alzheimer’s disease and other dementias. This research paves the way in which for novel intervention strategies which are finely tuned to the person’s neural profile, potentially transforming the therapeutic landscape for memory impairments and significantly improving patient outcomes.”
“These findings are exciting because they provide us insight into what is occurring within the brain between seeing something and saving it into memory,” says Wilma Bainbridge, assistant professor of psychology on the University of Chicago, who was not involved within the study. “The researchers listed here are picking up on a cortical signal that reflects what’s essential to recollect, and what will be forgotten early on.”
Lahner and Oliva, who can be the director of strategic industry engagement on the MIT Schwarzman College of Computing, MIT director of the MIT-IBM Watson AI Lab, and CSAIL principal investigator, join Western University Assistant Professor Yalda Mohsenzadeh and York University researcher Caitlin Mullin on the paper. The team acknowledges a shared instrument grant from the National Institutes of Health, and their work was funded by the Vannevar Bush Faculty Fellowship via an Office of Naval Research grant, a National Science Foundation award, Multidisciplinary University Research Initiative award via an Army Research Office grant, and the EECS MathWorks Fellowship. Their paper is published in .
