Neural networks often hide modular functions inside overlapping neurons
Neural networks are more modular than single neurons suggest
Artificial Intelligence
Summary
People usually think neural networks don’t have clear parts for different jobs because single neurons can do many things. The paper shows that even if individual neurons work on multiple tasks, the network can still separate those tasks in groups of neurons acting together. The authors found this hidden modularity in image, language, and cognitive task networks, showing that different functions can be isolated and manipulated without clear single-neuron separation. This helps us better understand and control how AI and brains organize their work.
What this means in practice
- •For ai model developers: Intervene on specific computations within complex neural networks even when individual neurons aren’t specialized, improving model interpretability and debugging.
- •For neuroscience data analysts: Use modular subspace decompositions to better align neural network features with brain activity patterns, improving brain-machine interface models.
Authors
Shreya Saha, Benjamin Bergen, Meenakshi Khosla
Abstract
Neural networks routinely develop representations in which individual neurons contribute to multiple distinct functions, leading to the widespread view that such networks lack modular organization. We challenge this conclusion by introducing a formally nested hierarchy of modularity: unit-level, which requires distinct computations to occupy disjoint neurons; orthogonal, which asks whether a rotation reveals functionally independent subspaces; and linear, which asks whether a bounded linear map can separate task-relevant computations into independent subspaces. We develop a unified framework for learning decompositions at each level and evaluating them causally through selective ablation. Across vision models, language models and multi-task Recurrent Neural Networks (RNNs), we find that networks which lack unit-level modularity nonetheless often exhibit clear functional modularity at the subspace level. In vision and language models, content versus style and syntax versus semantics respectively dissociate under subspace ablation despite sharing overlapping populations of neurons. These learned subspaces also align differentially with brain data, with content and style subspaces predicting higher and lower visual cortex respectively, and semantic subspaces outperforming syntactic ones in predicting language cortex activity. In RNNs trained on multiple cognitive tasks and previously shown to lack modular structure, our decomposition recovers sharp double dissociations between tasks. The absence of unit-level modularity frequently coexists with clear subspace-level modularity across domains, enabling targeted intervention on specific computations even in networks where no individual neuron is specialized, with broad implications for mechanistic interpretability and for understanding functional organization in both biological and artificial neural systems.