Flatclip provides middle-ground brain activity mapping with image encoders
FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning
Computational Engineering, Finance, and ScienceComputer Vision and Pattern Recognition
Summary
Brain activity is measured at different levels of detail, from broad regions to tiny voxels, but finer details need complex tools. The authors introduce FlatClip, which turns brain signals into flat images reflecting brain shapes, then uses an image model to understand them. This approach works better than simple regional methods on some tasks and shows that using brain anatomy helps improve analysis. FlatClip sits between coarse and fine brain mapping methods, offering a practical and efficient option.
What this means in practice
- •For neuroimaging analysis teams: Use flatclip to improve brain activity representations by integrating cortical geometry for better task performance without complex 3D models.
- •For medical imaging developers: Incorporate flatclip to decode visual and task-specific brain signals more effectively by focusing on relevant cortical regions.
Authors
Mo Wang, Wenhao Ye, Zihan Ning, Jiayu Zuo, Junfeng Xia, Hongkai Wen, Quanying Liu
Abstract
Recent fMRI foundation models differ substantially in the spatial scale at which they represent brain activity. ROI- and connectivity-based models are efficient but coarse, whereas voxel-level models preserve fine-grained spatial structure but require specialized 3D/4D architectures and costly fMRI-specific pretraining. We ask how effectively an image-pretrained encoder can reuse the spatial organization of cortical activity. Motivated by evidence that macroscale brain activity is strongly constrained by brain geometry, we introduce FlatClip, a frozen-encoder surface-level baseline that renders cortical activity as geometry-aware flatmap sequences and reuses a frozen SigLIP2 image encoder with only a lightweight downstream probe. Across resting-state benchmarks, FlatClip serves as a competitive middle-ground representation, outperforming ROI-level baselines on HCP and ADNI tasks while remaining weaker on PPMI and below the strongest voxel-level models overall. On visual-fMRI decoding, restricting the input to visual or NSD-provided task-active cortex improves performance, highlighting the value of task-relevant cortical coverage. Spatial perturbation controls reduce the predictive performance of flatmap features under both retrained and fixed readouts, and anatomy-linked arrangements consistently outperform vertex permutations across three colormaps. Together, these results position surface-level flatmap sequences as a practical middle-ground baseline between ROI and voxel models, and support the utility of anatomy-linked spatial organization for reusing image-pretrained features. Code is available at https://github.com/OneMore1/FlatClip.