Spherical interpolation improves compatibility of visual text search models
Spherical Interpolation for Backward-Compatible Multimodal Representations
Computer Vision and Pattern RecognitionMachine Learning
Summary
When different versions of AI models that match pictures and words are used, they make slightly different 'maps' for the same data, making it hard to upgrade without reprocessing everything. The authors found a way to 'blend' two versions of these maps along a curved path to better match old results without redoing all the stored data. This blending method works better than previous fixes and helps keep retrieval results consistent after upgrades. Their tests show this approach often finds better matches inside the blend than at either end.
What this means in practice
- •For image search engineers: Update multimodal search models without re-indexing the entire image gallery by adjusting new queries on a spherical blend to improve retrieval accuracy.
- •For recommendation system developers: Maintain consistency and improve compatibility across different versions of embedding models for item retrieval in multimedia recommendation engines.
Authors
Simone Ricci, Niccolò Biondi, Federico Pernici
Abstract
Contrastive vision-language models map visual and textual representations into a shared normalized embedding space, making cosine similarity the natural metric for cross-modal retrieval. A practical challenge arises during model upgrades: independently trained models generally produce incompatible representation spaces, so replacing a deployed model typically requires recomputing embeddings for the entire gallery, which is prohibitively expensive at scale. Orthogonal post-hoc alignment can partially mitigate this problem by mapping new-model queries into the old-model gallery space. However, because independently trained models can differ in fine-grained representation structure, the orthogonal alignment remains approximate, leaving a residual angular discrepancy between the old-model query and the aligned new-model query. We study whether interpolation along the spherical geodesic between these two normalized query representations can improve retrieval without re-indexing the gallery. We characterize when this path contains an interior query direction closer to an idealized retrieval-optimal direction than either endpoint, and connect this characterization to Recall@$K$ through a local margin-based certification result. Experiments across multiple benchmarks and model families show that post-alignment spherical interpolation improves over orthogonal alignment alone, recovering backward-compatibility in most evaluated settings. Consistent with our geometric characterization, per-query oracle analysis shows that retrieval-favorable interior points occur frequently in practice. Code is available at https://github.com/miccunifi/SLERP_backward_compatibility .