Frozen natural image autoencoders match fMRI models in brain trait prediction

Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction

Computer Vision and Pattern Recognition

Summary

Understanding brain activity using fMRI usually requires training special AI models on lots of brain data, which is expensive and time-consuming. The authors show that using an AI model trained only on natural images, without any specific brain training, can predict personal traits and brain states from fMRI data just as well as specialized models. This suggests simpler, cheaper ways to analyze brain scans without the need for heavy fMRI-focused training. Their approach might help researchers and developers compare new brain analysis methods to a strong baseline.

What this means in practice

  • For medical imaging teams: Use frozen natural image autoencoder features to predict patient traits from resting-state fMRI data without costly fMRI pre-training.
  • For brain computer interface developers: Apply temporal models on frozen autoencoder features to decode brain states from task fMRI for improved signal interpretation.

Authors

Juhyeon Park, Yeonwoo Kim, Peter Yongho Kim, Yansen Wang, Mingqing Xiao, Dongqi Han, Dongsheng Li, Taesup Moon

Abstract

Foundation models pre-trained on large-scale fMRI datasets have shown strong downstream performance, but at substantial data and computation cost. To investigate how much fMRI-specific pre-training is actually needed for such performance, we introduce FReD, which derives fMRI representations from a frozen Deep Compression AutoEncoder (DCAE) pre-trained exclusively on natural images and pairs them with a task specific readout. For trait prediction, FReD summarizes frame-wise representations by their temporal mean and log-standard deviation and applies linear probing, with late fusion across two normalization schemes. For state prediction, it represents each frame as a single token and models temporal dependencies with a shallow Transformer. Across four resting-state datasets spanning six trait-prediction targets, linear probes on frozen DCAE features generally outperform those on fMRI foundation model representations and remain competitive with fully fine-tuned fMRI foundation models. On three task-fMRI state-prediction tasks, a temporal readout on DCAE features performs comparably to the strongest foundation models evaluated. A Gaussian injection analysis further shows that localized signal changes are recovered more accurately from the frozen DCAE features than from the evaluated foundation-model representations. Together, these results show that strong performance on current fMRI benchmarks is possible without fMRI-specific representation pre-training, making frozen natural-image features as a useful baseline for assessing its added value.