EEG brain signals aligned with images using adaptive cortical approach
Adaptive Cortically Constrained EEG-Vision Alignment for Zero-Shot Brain-to-Image Retrieval
Computer Vision and Pattern Recognition
Summary
Matching brain signals recorded by EEG to the images a person sees is challenging because EEG data is noisy and mixed across the scalp. The authors developed a method that first maps signals to specific brain regions and then uses a smart way of adjusting how much visual guidance to provide based on the quality of each signal. Their approach improves the ability to find matching images without prior examples of the exact image seen. This could help better understand how brain activity corresponds to visual perception.
What this means in practice
- •For brain-computer interface developers: Improve brain-to-image retrieval systems by adapting EEG signal mapping to brain regions and customizing visual feedback per signal quality.
- •For neurotechnology hardware makers: Design EEG devices optimized for capturing source-level brain activity patterns for image decoding tasks.
Authors
Ye Wang, Haokun Ren, Wei Wu, Guoyin Wang, Zhuliang Yu, Hong Yu, Ke Liu
Abstract
Zero-shot brain-to-image retrieval requires robust alignment between noisy EEG responses and visual representations. Existing EEG-vision alignment methods often operate in sensor space and apply fixed visual supervision to all responses, ignoring both spatial mixing in scalp EEG and response-wise variability in alignment reliability. We propose an adaptive cortically constrained EEG-vision alignment method for zero-shot brain-to-image retrieval. The method reconstructs EEG responses into predefined ROI-level source-pattern representations and encodes them with a Neuro-ROI Attention Encoder. To handle response-wise variability, we introduce an evidence-based adaptive visual supervision strategy that weights detail-controlled visual targets using model-based alignment evidence. On THINGS-EEG, the proposed method achieves strong 200-way zero-shot retrieval performance, with ROI-level attribution providing post hoc interpretability of the learned source-pattern representations. These results show that cortically constrained representation learning and adaptive supervision can jointly support EEG-vision alignment for zero-shot brain-to-image retrieval.