SCORE: Subject Coordinate Recovery for Label-Free Cross-Subject EEG-to-Image Retrieval
2026-08-19 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied how to better interpret brain signals (EEG) to identify images people see, even for new users without extra training data. They found that different people's brains represent visual concepts similarly but in different 'coordinate directions.' To fix this, they developed SCORE, which aligns brain signals from various users without needing labeled data from new users. Their method improved accuracy significantly on public tests and works without changing the model or needing new user data, making brain-based image decoding more practical for real-world use.
EEGvisual decodingimage retrievalcoordinate alignmentcross-subject generalizationunsupervised learningorthogonal transformationhubness correctionbrain-computer interfaceTHINGS-EEG2 dataset
Authors
Zhenyao Cui, Siyuan Kan, Siyang Li, Ziwei Wang, Dongrui Wu
Abstract
Accurate visual decoding can reveal how the brain represents visual information and recover perceived content from neural signals such as electroencephalography (EEG), with potential for neural communication. However, current EEG-to-image retrieval methods perform far below their within-subject counterparts for new users without labeled calibration, limiting real-world deployment. To understand this gap, we analyze EEG features across subjects and find that different subjects preserve similar relationships among concepts but express them along different coordinate directions. We therefore propose Subject Coordinate Recovery (SCORE), a target label-free framework combining recovery-aware source training with coordinate alignment at deployment. During training, SCORE aligns source subject EEG with a common image space and simulates unseen-subject recovery through source-only episodes. At deployment, with both encoders frozen, SCORE selects reliable EEG-image landmarks through hubness-corrected matching and estimates an orthogonal transformation to recover target EEG coordinates without source data or target labels. In 200-way retrieval on two public benchmarks, SCORE outperforms the unadapted baseline for every target subject and achieves the best overall accuracy. It reaches 53.23%/83.55% and 12.01%/32.16% Top-1/Top-5 on THINGS-EEG2 and Alljoined-1.6M, respectively, surpassing the strongest baselines by 17.45/15.70 and 3.08/4.62 percentage points. Without target labels or encoder updates, SCORE brings brain-based visual decoding closer to robust, practical, low-latency deployment across users.