Hvmap enables federated search across different vector databases
Federated Vector Retrieval across Heterogeneous VectorDBs
DatabasesInformation Retrieval
Summary
Many AI systems look for information using special databases that store data as vectors, but these databases often use different methods to create these vectors, making it hard to search across them all at once. The authors present HVMap, a tool that learns how to translate between these different vector systems so queries can work across them without rebuilding everything. This means faster and cheaper searches that still give good results even when the databases are very different. The tool works with any database system and can handle many databases at once.
What this means in practice
- •For enterprise search teams: Enable unified search queries over multiple independently built vector databases without rebuilding or re-indexing the databases.
- •For cloud database providers: Offer federated vector search services that handle heterogeneous vector databases efficiently to increase throughput and reduce setup time.$Commercial implications: Supports selling scalable federated search tools that integrate diverse vector databases for enterprises needing quick deployment.
Authors
Mintaek Lim, Sieun Hyeon, Geon Chio, Jaeyoung Do
Abstract
Vector databases are now core infrastructure for AI retrieval systems, yet many real deployments must serve queries over multiple independently constructed vectorDBs built with different embedding models. This creates a systems problem of federated retrieval over heterogeneous vectorDBs: vectors and similarity scores are not directly comparable across embedding spaces, while fully resolving this incompatibility through corpus-wide re-embedding and re-indexing is often operationally prohibitive. Existing methods therefore face a difficult trade-off between retrieval quality, construction cost, and query throughput. We present HVMap, a lightweight compatibility layer for federated retrieval across heterogeneous vectorDBs. Instead of rebuilding databases into a unified space, HVMap learns bidirectional mappings between embedding spaces using a fuzzy contrastive loss, enabling calibrated cross-database scoring without corpus migration or index reconstruction. HVMap is system-agnostic and further scales to multi-vectorDB settings while preserving ranking fidelity through a hub-based architecture. We evaluate HVMap in open-domain, cross-domain, and scale-out settings, and also construct a new benchmark for heterogeneous cross-domain federation. Across these settings, HVMap consistently achieves a better quality--cost--throughput trade-off than baselines, achieving up to 3.59X higher nDCG@10, up to 2.01X higher query throughput, and construction time as low as 0.3% of baseline methods. These results show that HVMap makes heterogeneous vectorDB federation practical at scale.