Human brain networks improve multimodal AI model performance across tasks
The Platonic brain bridge hypothesis: human brain networks as an architectural prior for omni models
Machine Learning
Summary
The paper explores how artificial intelligence models that process video, audio, and text together can develop internal structures similar to the human brain. The authors show that these 'omni models' tend to have brain-like patterns, and these similarities work in both directions: the models reflect brain activity, and brain structures can help improve the models. By assigning parts of AI models to mimic specific brain networks, performance improves significantly on various tasks. This suggests that understanding how the brain organizes information can guide building better AI systems that handle multiple types of data.
What this means in practice
- •For ai developers: Improve multimodal AI models by structuring them based on human brain networks to boost accuracy across varied tasks.
- •For neurotechnology engineers: Build brain-inspired AI components that better align with neural activity patterns for enhanced human-computer interaction systems.
Authors
Pengfei Zhang, Biao Tian, Xiangang Li, Li Liu
Abstract
We propose the Platonic brain bridge hypothesis: omni models, which process video, audio and text jointly like the brain, converge on brain-like representations, and the correspondence is bidirectional. From model to brain, brain-likeness of seven omni models is stable across participants, and our encoding models on their internal hidden states rank first on the Algonauts 2025 out-of-distribution leaderboard. From brain to model, three contributions follow. Brain-MoE gives seven cortical networks one brain-pretrained expert each and raises held-out accuracy in all 15 model-benchmark pairs by 6.42 percentage points on average. Brain-AVQA builds questions from video clips labelled by the most responsive brain network; the real network-to-expert map exceeds shuffled maps in-domain on all three models. Brain-Scope uses sparse autoencoders to localize the correspondence to a small subset whose removal weakens brain prediction in all three bases tested. Human brain networks are therefore a usable architectural prior for omni models.