EEG signals guide faster vision language answers on cluttered images
BrainFocus: EEG-Guided ROI Selection for Efficient Vision-Language Models
Computer Vision and Pattern Recognition
Summary
Answering questions about big, messy pictures is slow because computers look at the whole image even if only part of it matters. The authors found a way to use brain signals recorded when a person sees an image to guess what part is important. Their method only sends that important part to the vision-language model when confident, saving time but still giving good answers. They show this works well in tests with many images and questions.
What this means in practice
- •For mobile app developers: Build apps that provide quick answers to questions about busy photos by focusing computational work on important parts identified via EEG signals.
- •For neuromarketing teams: Use EEG to quickly identify visual elements that consumers notice in cluttered advertisements, improving targeted content analysis.
Authors
Yihui Peng, Guorui Lu, Qinyu Chen
Abstract
Vision-language models (VLMs) achieve strong visual question answering (VQA) performance, but processing large cluttered images is computationally expensive when only a small region is relevant. Electroencephalography (EEG) signals, which capture human neural responses to visual stimuli, can provide a human-derived semantic cue about the region of interest (ROI). However, EEG-guided visual category decoding remains imperfect, making direct ROI routing unreliable. In this work, we propose BrainFocus, a reliable EEG-guided efficient VLM framework for VQA. An EEG classifier predicts a target category, and a YOLO detector localizes the matching ROI. The VLM receives the cropped ROI only when both predictions pass confidence thresholds; otherwise, it processes the full image. For evaluation, we build on EEG-ImageNet to construct a 40-class benchmark comprising generated cluttered images and real object-centric images, with target-ROI annotations and 600 English visual question-answer pairs. Across Qwen3.5-VL 2B, 4B, and 9B models, BrainFocus improves VQA accuracy by 4.14-9.87 percentage points (pp) on cluttered scenes while reducing input tokens and total tokens by 23.2%-39.4% and 23.2%-39.3%, and end-to-end floating-point operations (FLOPs) by 23.2%-39.5%. These results demonstrate that EEG can guide efficient VLM inference even when its semantic decoding is imperfect.