Chain on demand improves long-term conversational memory accuracy
Chain on Demand: Deferred Evidence Organization for Long-Term Conversational Memory
Computation and LanguageArtificial Intelligence
Summary
Remembering long conversations is tricky because it's hard to organize past chat details before knowing what questions might come next. The authors created Chain on Demand, a way to pick and organize important past conversation bits only when a new question is asked. This method builds a temporary chain of key points relevant to the question, without changing the original memory. Their tests show it answers questions more accurately by waiting to organize information until it’s really needed.
What this means in practice
- •For virtual assistant developers: Enhance virtual assistants to retrieve long conversation details more accurately by organizing evidence only when answering specific user queries.
- •For customer support platform teams: Improve customer chat history access by dynamically assembling relevant conversation turns at question time without altering stored logs.
Authors
Di Lu, Zechen Li, Hao Xue
Abstract
Long-term conversational memory faces a timing mismatch: write-time summaries and graphs organize history before a future query is known, while independent retrieval can allocate a limited evidence budget to redundant or fragmented turns. We introduce Chain on Demand (CoD), a query-time evidence organization framework over an unchanged turn-level memory. Given a broad candidate pool, CoD constructs a temporary evidence chain by allocating a fixed anchor budget according to query relevance, lexical diversity relative to selected anchors, and newly covered query terms, followed by deterministic restoration of each selected anchor's same-session neighborhood. The resulting chain is used only for the current query, while the underlying conversational memory remains unchanged. In a paired comparison over all 1,986 LoCoMo questions, CoD achieves 82.2% accuracy on 1,540 non-adversarial questions and 76.3% on all 1,986 questions. On LongMemEval-S, one fixed configuration that indexes both user and assistant turns obtains 86.6% across all 500 questions. These results show that long-term memory can preserve the original conversation while deferring evidence organization until the information need is known.