Decentralized vector search system boosts accuracy and cuts network use
Semord: Learned Semantic-Preserving Placement and Low-Fanout Routing for Distributed Vector Search
Networking and Internet Architecture
Summary
Vector databases store information as points in space to help computers find similar data quickly. Existing systems often rely on a central tracker to know where data lives, but this can cause problems if parts disconnect or move. The authors propose Semord, which spreads out data smartly across many peers without a central coordinator and finds similar data by asking only a few relevant peers. Their design improves how accurately queries find good matches and reduces the number of peers contacted, saving communication and memory.
What this means in practice
- •For distributed database engineers: Build decentralized vector search services that maintain high accuracy without relying on a central coordinator, improving efficiency under changing network membership.
- •For cloud infrastructure teams: Create scalable, resilient vector search overlays for multi-site deployments that reduce memory needs and network traffic when processing similarity queries.
Authors
Shengze Wang, Yi Liu, Yifan Hua, Xiaoxue Zhang, Chen Qian
Abstract
Vector databases are increasingly deployed in distributed settings where different users, sites, or domains maintain vector data. Existing vector databases rely on a coordinator to record which shards store which parts of the vector space and to route each query to those shards. In a decentralized setting, peers may join, leave, or move data without a trusted node tracking every change, and outdated routing information can therefore send queries to the wrong peers or require contacting many peers, reducing vector retrieval recall and increasing network latency. We present Semord, a decentralized vector search overlay system that achieves high recall by routing each ANN query to a small set of relevant peers, without relying on a centralized coordinator. Semord addresses this problem by making semantic locality routable: 1) We propose VHash to place semantically related vectors near each other in the overlay key space while avoiding load imbalance, so that each query only needs to contact a small neighborhood of peers for distributed local ANN ranking. 2) We design VecDHT, a communication protocol that maintains decentralized routing, region metadata, churn resilience, and VHash updates under membership and workload changes. Our extensive experiments on a real testbed show that Semord improves recall by more than 15% and reduces contacted peers by over 60% compared with decentralized baselines. Semord also approaches the recall and latency of a centralized oracle baseline while reducing peak peer-local ANN index memory by more than 2X. Controlled large-scale simulations further show that Semord scales across real-world embedding workloads and remains robust under churn for scoped vector retrieval as a decentralized overlay.