Braintaskonomy improves brain scan learning by organizing task transfers
BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models
Computer Vision and Pattern Recognition
Summary
Brain scans called fMRI come from many different situations, and training computers to understand them usually treats all data the same. The authors show that measuring how different brain scan tasks and data types help each other can guide smarter learning steps. Their method organizes training and adaptation in a way that improves accuracy on brain data by choosing what to learn first and how to transfer knowledge between tasks. This helps computers better analyze brain functions across diverse scanning conditions.
What this means in practice
- •For medical imaging teams: Enhance fMRI analysis pipelines by selecting pretraining and transfer tasks that improve prediction accuracy across varied brain scan datasets.
- •For machine learning engineers: Design brain imaging models that use measured task relationships to optimize training and adaptation for better performance under limited data budgets.
Authors
Junfeng Xia, Wenhao Ye, Junxiang Zhang, Jiayu Zuo, Mo Wang, Quanying Liu
Abstract
fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are commonly treated as a flat mixture and downstream tasks are adapted independently. We study whether measured learning relations can organize both stages without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates difficulty and directed facilitation across ten fMRI domains, yielding a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer across fifteen tasks constructs a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5%, respectively, relative to uniform sampling over both dimensions, and shows strong downstream performance across six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, while exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available than for matched random controls. Together, these findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.