Neural Link Between Seeing and Imagining Revealed in New Study

A study published in the journal Science has provided neural-level evidence that the same neurons and neural code are activated when a person sees an object and when they later imagine it. The research, involving patients with implanted electrodes, found that approximately 40% of neurons that respond to visual perception reactivate during imagination. Researchers suggest the findings could advance the understanding of visual processing and inform future technologies.

Study Methodology and Key Findings

The research was conducted by a team led by Varun Wadia of Cedars-Sinai Medical Center and the California Institute of Technology. The study involved 16 patients with epilepsy who had electrodes temporarily implanted in their brains for medical monitoring.

Procedure:
In the first phase, participants viewed hundreds of images from categories including faces, animals, plants, words, and small objects. Researchers recorded the activity of over 700 individual neurons in the ventral temporal cortex, a brain region involved in object recognition. They noted which neurons fired in response to each image and their firing patterns.

Imagination Task:
In a second phase, participants closed their eyes and imagined objects they had previously seen while researchers monitored the same neurons.

Primary Result:

The study found that approximately 40% of the neurons that were active during visual perception also reactivated when the participant imagined the same object, with a similar strength of activation.

Pattern Specificity:
The neural activation patterns were specific enough that researchers could identify which object a participant was imagining. According to the researchers, these patterns could also reveal details about the imagined object's properties, such as its size, angle, and context.

Scientific Context and Verification

The findings provide direct evidence at the level of individual neurons for a concept previously suggested by brain imaging studies, such as functional MRI, which indicated overlapping neural circuits for vision and imagination but could not track single-neuron activity.

• The research builds on earlier work by study co-author Doris Tsao, a professor at the University of California, Berkeley, on how the visual system recognizes faces and objects.
• According to the study, advanced artificial intelligence tools were used to decode the neural activity. The team utilized generative AI to create novel images based on the discovered neural code and verified that the brain's responses to these AI-generated images aligned with predictions.

Researcher Perspectives and Potential Applications

Researchers not involved in the study commented on its implications.

Kalanit Grill-Spector, a psychology professor at Stanford University, stated that this demonstration at the neural level had not been achieved before. She noted these insights could help scientists build computer models to simulate vision and vision disorders like macular degeneration, which could potentially aid in developing prosthetic devices to restore sight.

Thomas Naselaris, a neuroscientist at the University of Minnesota, said the research helps explain how the brain uses imagination to augment visual information, such as modeling unseen parts of three-dimensional objects or assembling familiar objects into new configurations.

Ueli Rutishauser, whose lab at Cedars-Sinai conducted much of the work, stated that the activation patterns could reveal specific details about imagined objects.

Limitations and Areas for Future Research

The study does not explain the neural mechanisms in individuals with aphantasia, a condition where people cannot voluntarily summon mental images.

• Rutishauser reported learning about the condition when a scientist told him, after a presentation of the research, "I don't see anything when I close my eyes."
• Scientists have suggested that people with aphantasia may use words or concepts to recall visual information, but studies of their neurons would be needed to understand the underlying neural processes.