Human visual cortex feature preferences revealed with receptive field constrained images
Receptive-field-constrained stimulus optimization for human early and intermediate visual cortex
Machine Learning
Summary
Understanding what patterns specific brain areas respond to is hard because some parts of the brain look at tiny regions of what we see. The authors created two new ways to design images that strongly activate small parts of the early human visual brain using brain scan data. These images reveal the types of features like edges, colors, and textures that these brain areas prefer. Their work helps map how the brain processes visual details in a more precise, data-driven way.
What this means in practice
- •For clinical neuroimaging teams: Generate detailed stimulus images to better interpret functional brain scans of early visual areas in patients with vision disorders.
- •For computer vision engineers: Design image features inspired by human early visual cortex selectivity to improve local image recognition algorithms.
Authors
Junru Zhao, Hanfei Guo, Andrew Luo, Margaret M. Henderson
Abstract
An ongoing challenge in sensory neuroscience is to characterize the feature dimensions encoded by cortical populations. Recent approaches probe feature selectivity in a data-driven way, by synthesizing a most-exciting-input (MEI) for a target neural population. While this approach has been successfully applied to human higher visual cortex using fMRI data, generating MEIs for early- and mid-level retinotopic visual areas requires additional modeling constraints due to small receptive field sizes. To address this challenge, we introduce two novel MEI generation frameworks, Receptive Field Diffusion for Visual Exploration (RF-DiVE) and Receptive Field Gradient Optimization (RF-GO). Both methods use a population receptive field (pRF)-constrained voxelwise encoding model; RF-DiVE combines this with a pretrained latent diffusion model, while RF-GO uses regularized gradient ascent. When applied to single voxels in retinotopically defined areas V1-hV4, using data from the Natural Scenes Dataset, we obtain MEIs that exhibit consistent structure within the pRF, suggesting selectivity for local features like contour, color, and texture. We systematically compare MEIs generated by RF-DiVE and RF-GO using two encoding backbones, performing in-silico validation of predicted responses to MEIs using independent encoding models. Across all methods and all visual areas, MEIs elicit higher model-predicted responses than the most activating natural images. We further find that the choice of generation framework and encoding backbone differentially affects MEI properties, including their visual appearance, structural interpretability, and cross-model generalizability. These results offer a new approach for performing data-driven characterization of spatial and feature selectivity across human visual cortex.