Semantic correspondence reveals shared geometry in neural networks
The Geometry of Meaning in Neural Manifolds
Artificial Intelligence
Summary
Different neural networks often represent the same ideas in completely different ways, making it hard to compare or combine them. This paper shows that by finding connections between parts that mean the same thing in different models, we can line them up and understand their shared structure. This lets us transfer knowledge between models without retraining, work across different types of AI systems, and even relate brain activity patterns across people. The authors demonstrate this idea across many tasks, network types, and data, highlighting a hidden shared geometry beneath the surface differences.
What this means in practice
- •For machine learning engineers: Enable zero-shot transfer of knowledge between independently trained models by aligning their semantic elements without retraining.
- •For neurotechnology developers: Use shared geometric structures in brain activity across individuals to improve neural decoding and brain-computer interfaces.
Authors
Valentino Maiorca
Abstract
Just as different languages express the same concept through different words, neural networks express similar functions through incompatible coordinate systems. Different architectures, random initializations, and optimization trajectories produce incompatible parameterizations in extrinsic coordinates, obscuring functional relationships between independently trained models. When models independently solve similar tasks on related data, do their learned representations share exploitable structure despite incompatible coordinates? This thesis shows that semantic correspondence is a fundamental primitive revealing shared geometric structure beneath incompatible parameterizations. By identifying which elements refer to the same semantic content, correspondence breaks the symmetry of arbitrary implementation choices, anchoring incompatible systems to a shared semantic frame and revealing the intrinsic geometry that enables zero-shot transformation between independently trained models. Building on this principle, fourteen contributions explore correspondence-based alignment across zero-shot model stitching, cross-modal translation, policy transfer, activation steering, and neural decoding. They span domains (vision, language, reinforcement learning, neuroscience), architectures (CNNs, Transformers, graph networks), and scales (from small supervised models to billion-parameter generative systems). Each exploits different geometric properties (angular, metric, spectral, sparse, or distributional), yet all instantiate the same principle. It extends beyond artificial systems: fMRI data show that different human brains processing identical stimuli exhibit alignable geometric structure. By revealing intrinsic geometric structure beneath extrinsic parameterization, semantic correspondence transforms model isolation into interoperability.