Dual-path speech retrieval boosts voice assistant accuracy and speed
DUPAR: Dual-Path Conversational Retrieval via Speech Retriever with Cross-Turn Evidence Caching
Information Retrieval
Summary
Voice assistants often turn spoken questions into text before searching for answers, which can be slow and cause mistakes. The authors propose DUPAR, a new system that quickly checks recent conversation parts for answers using sound patterns, and only searches more broadly when needed. This approach is faster and more accurate, especially with noisy or varied speech, by remembering what was discussed before and focusing searches effectively. DUPAR improves retrieval accuracy and speeds up response time compared to traditional methods.
What this means in practice
- •For voice assistant developers: Speed up and improve accuracy of voice assistants answering domain-specific queries by caching conversational evidence and using a dual-path retrieval system.$Commercial implications: Enables products that offer faster, more reliable voice-based search over specialized knowledge, improving user experience for digital assistant software.
- •For call center automation teams: Enhance automated phone systems by reducing errors and response times when fetching relevant information from support databases during conversations.
Authors
Yuanjun Li, Yiwen Liu, Dapeng Li, Zhiwei Xu, Bin Zhang, Shengtao Zhang, Rong Shen
Abstract
Voice assistants grounded in external knowledge typically use automatic speech recognition (ASR) to transcribe speech queries before retrieving evidence from textual knowledge bases. This cascade adds latency and propagates recognition errors, whereas direct speech retrieval is vulnerable to cross-modal misalignment. To address these limitations, we propose DUPAR, a conversational retrieval framework with complementary slow and fast paths. The fast path uses a task-adapted audio encoder aligned with frozen BGE-M3 text embeddings to search a cross-turn evidence cache. When cache confidence is insufficient, the slow path fuses full-index retrieval using audio and ASR-transcript embeddings, and the selected evidence refreshes the next-turn evidence cache through one-hop graph expansion. On a domain-specific knowledge base, our trained audio encoder approaches text-retrieval accuracy on clean speech with a 3.75$\times$ query-side speedup over ASR + Text Encoder. It raises average Recall@10 from 0.771 to 0.875 on the noise benchmark and improves overall Recall@1 by 4.2 percentage points across synthesized speaking styles. Compared with full-index audio retrieval, cross-turn evidence caching significantly reduces retrieval errors when the previous turn retrieves correct evidence and the follow-up targets a one-hop neighboring chunk.