SUP-MCRL: Subject-aware Unified Pseudo-feature Coded Multimodal Contrastive Representation Learning for EEG Visual Decoding
2026-06-15 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address a problem in brain-computer interfaces where recognizing what a person sees from brain signals becomes less accurate with natural images. They created a new system called SUP-MCRL that combines three parts: one that finds important visual details without extra models, another that makes brain signal reading stronger across different people, and a third that keeps learned features stable over time. Their approach improved recognition accuracy compared to previous methods, especially when tested without prior training on specific subjects. They tested their method on a dataset called THINGS-EEG and shared their code for others to use.
brain-computer interfacevisual decodingcontrastive learningsemantic consistencyEEGspatial attentionmulti-scale convolutionzero-shot learningcross-subject generalizationTHINGS-EEG dataset
Authors
Shengyu Gong, Weiming Zeng, Yueyang Li, Zijian Kang, Hongjie Yan, Wai Ting Siok, Nizhuan Wang
Abstract
Non-invasive brain-computer interfaces suffer severe fidelity degradation in neural visual decoding when generalizing to natural visual experiences. Conventional multimodal contrastive representation learning solely optimizes geometric distance alignment, neglecting semantic consistency and subject selectivity, causing spurious zero-shot alignment. We propose SUP-MCRL, a unified framework integrating three collaborative mechanisms: (1) Semantic-entity Aware Visual Encoder (SAVE), learning spatial attention to extract semantic content without pre-trained saliency models; (2 Unified EEG Enhancer (UEE), employing multi-scale atrous convolutions and inter-band attention for adaptive cross-subject robustness; and (3) Prototype-based Progressive Augmenter (PPA), maintaining an EMA-updated pseudo-feature pool to prevent representation collapse. Zero-shot experiments on THINGS-EEG achieve 66.0%/91.9% (Top-1/Top-5) intra-subject and 24.0%/52.9% LOSO accuracy, surpassing state-of-the-art methods. Code is available at https://github.com/NZWANG/SUP-MCRL.