TurboVec: A Case Study in Cost-Efficient Private Retrieval for Enterprise RAG via Codebook-Oblivious Quantization
Machine LearningArtificial IntelligenceInformation Retrieval
Summary
The authors address challenges in systems that retrieve information based on vector search, especially about privacy leaks and accuracy after filtering for multiple users. They introduce TurboVec, a vector index using TurboQuant, which does not need training on data and avoids revealing sensitive dataset patterns. On benchmarks, TurboQuant performed better than trained alternatives in retrieval accuracy while using less memory than some popular methods. When deployed in real settings, TurboVec also showed fast query responses and maintained good accuracy with privacy-preserving filtering methods. The study notes limitations such as testing on only one dataset and using synthetic data for privacy evaluation.
Authors
Navnit Shukla, Kamal Pandey, Omsankar Tiwari
Abstract
Retrieval-Augmented Generation (RAG) systems increasingly power enterprise LLM applications, yet the vector retrieval layer introduces two underexplored challenges: (1) trained codebook quantizers may expose corpus statistics during index construction, creating a leakage channel in multi-tenant deployments, and (2) post-hoc filtering for tenant isolation degrades recall on selective queries. We study TurboVec, an open-source vector index built on TurboQuant - a codebook-oblivious scalar quantizer requiring no corpus-dependent training. On the DBpedia OpenAI embeddings benchmark (d=1536, 100K-999K vectors), TurboQuant 4-bit outperforms trained FAISS Product Quantization at the same memory budget by 8.5-8.9 percentage points in Recall@5 across all scales. Compared to HNSW (R@5=0.991) and IVF-PQ (R@5=0.840), TurboQuant occupies a distinct design point: higher recall than IVF-PQ without training, at 4-8x less memory than HNSW. Deployed on Snowpark Container Services, TurboVec achieves 11ms median query latency at 100K vectors versus 707ms for warehouse brute-force scan. Kernel-level allowlist filtering maintains 0.86-0.93 Recall@10 across 10-1000 tenant workloads versus 0.09-0.19 for post-filter baselines. Codebook-oblivious design reduces membership inference accuracy to near-random (50.0%) versus 57.3% for PQ codebooks. Limitations include single dataset evaluation, uncompressed HNSW comparison, and privacy evaluation on synthetic data only.