Vision and language models represent children’s brain responses differently than adults
Vision Models Capture Complementary Representations in Children That Converge in Adults
Computer Vision and Pattern Recognition
Summary
This paper studies how different artificial vision models match brain activity from children and adults when they see images. The authors found that image-text aligned models, like CLIP, match children's brain patterns well, while vision-only models line up better with adult brain patterns. Both types capture different, complementary ways the brain processes images and meanings as kids grow up. The study helps us understand how visual understanding evolves with age and could guide AI that learns more like humans.
What this means in practice
- •For computer vision engineers: Develop vision systems that incorporate complementary representations from image-text and vision-only models to improve performance across varied developmental stages.
- •For neurotechnology developers: Create brain-computer interfaces or diagnostic tools that consider developmental differences in visual processing by leveraging model-brain alignment insights from children and adults.
Authors
Domenic Bersch, Timothy Schaumlöffel, Michela Proietti, Antonia Franaszek-Traczewska, Hannah Elisabeth Zwad, Siying Xie, Marlena Baldauf, Teresa Sylvester, Christina Maria Schätz, Bert Turtleton, Moritz Köster, Stefanie Höhl, Radoslaw Cichy, Gemma Roig
Abstract
How the relationship between image-text aligned and vision-only models changes across development remains unknown. We applied time-resolved representational similarity analysis to EEG data from 8-year-olds, 12-year-olds, and adults. CLIP showed significant model-brain alignment already in children, whereas vision-only models aligned more strongly later in development. Residual analyses revealed complementary structure, with CLIP retaining alignment beyond an architecture-matched vision-only model from age 8. Much of the model-brain alignment reflected semantic structure, with EEG representations of category and animacy strengthening toward adulthood, while alignment with low-level image properties remained weak across age. Together, these findings suggest that image-text aligned and vision-only models capture complementary aspects of developing visual representations and may provide useful constraints for AI systems that aim to learn and adapt over time.