Displacement vectors reveal shared reality across AI models
Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities
Machine Learning
Summary
Different AI models trained independently don’t share exactly the same internal representations for the same inputs, but the directions from one input’s representation to another’s are very similar. The authors found that although the exact positions differ, the relationships or differences between concepts are aligned between many models, including vision and language ones. This means models share a kind of underlying 'directional' understanding of concepts, even if their individual concept points don’t match perfectly. The paper also shows new ways to transfer model capabilities between AI systems without retraining by leveraging these shared directions.
What this means in practice
- •For machine learning engineers: Transfer complex model capabilities between different AI systems without fine-tuning by using cached forward passes of source models.$Commercial implications: Enables new software products for capability sharing that reduce compute and speed up AI deployment.
- •For ai system integrators: Improve multimodal AI system design by aligning semantic features across vision and language models to enhance interoperability.
Authors
Chenming Shang, Yujin Tang, Jun Jie Ou Yang, Ruize Xu, Adam Breuer, Nikhil Singh
Abstract
The Platonic Representation Hypothesis (PRH) claims that independently trained models converge on a shared statistical model of reality, yet recent work finds only weak pointwise similarity between models. In this paper, we show that what models share is not the location of samples in representation space, but the directions (displacement vectors) between them. Under a single orthogonal alignment--rotation and reflection only--these displacement vectors are substantially preserved across 44 independently trained vision and language encoders spanning modalities and asymmetric capability pairs, consistent with the PRH evidence. The samples' absolute positions are not, consistent with recent counter-evidence. Both arise from a single decomposition: representations split into a shared semantic component that is linearly aligned across models, and a private capability component that is not. We trace this geometry to concept-level structure: within a model, parent concepts are orthogonal to their child variation vectors; across models, concept displacements are parallel. Our theory falsifiably predicts (and experiments confirm) that fine-tuning preserves pointwise similarity but collapses displacement, and that relational distillation does the opposite. A major implication is that, because semantics align linearly but capabilities do not, capabilities can be imported from one model to another using a single cached forward pass through the source. We call this Shadow Casting. As a proof of concept, our SHADOWCLIP instantiation outperforms strong fine-tuned baselines at orders of magnitude less compute. A cache can be released alongside open model weights, letting one model's capabilities be downloaded and imported into any number of other models without fine-tuning.