Structured visual targets improve eeg image retrieval accuracy across people

Structured Visual Target Learning For Cross-Subject eeg-to-image retrieval

Computer Vision and Pattern Recognition

Summary

Matching brain signals (EEG) to images is hard because brain signals vary a lot between different people. The authors focus on improving how images are represented so that a system trained on some people can better understand brain signals from a new person. They do this by organizing visual information into meaningful parts and learning to match these with brain signals during training. Their method works better than previous ones, especially after a refinement step that improves alignment without retraining.

What this means in practice

  • For brain computer interface developers: Build more accurate systems that decode unseen users' brain signals into matching images using structured visual target learning without retraining encoders.$Commercial implications: Enables commercial EEG-based image retrieval products that generalize well to new users, improving user experience and reducing customization costs.
  • For machine learning engineers: Integrate structured visual target representations to improve cross-subject alignment in EEG-to-image retrieval models for applied neuroscience tools.

Authors

Salini Yadav, Taveena Lotey, Mickaël Coustaty, Pravendra Singh, Partha Pratim Roy

Abstract

Cross-subject EEG-to-image retrieval requires a neural represen- tation trained on source subjects to remain aligned with a visual embedding space for an unseen subject. Whereas existing methods primarily focus on the EEG side, we address this problem from the perspective of the visual target. Our approach preserves the spatial information of the Perception Encoder, converts its patch grid into a compact set of learned visual views, and aggregates them for each image with a block-structured, content-dependent router. The target is learned jointly with the EEG encoder through contrastive learning with MMD regularization across source subjects. For deployment, we propose a training-free representation refinement that aligns frozen embeddings without updating either encoder. Under leave- one-subject-out evaluation on THINGS-EEG2, the structured target achieves 35.3%/65.6% Top-1/Top-5 accuracy, the best among com- pared methods. Refinement raises this to 48.1%/77.1%, an 18.5% Top-1 gain over the strongest compared method, improving all ten held-out subjects.