Scalaris improves memory efficiency for enterprise conversational AI assistants
Scalaris: A Scalable Enterprise Memory Architecture for Conversational AI Assistants
Computation and LanguageArtificial Intelligence
Summary
AI assistants that talk to people in companies need to remember details about users and past conversations to work well. The authors designed Scalaris, a system that stores and retrieves these memories more efficiently by treating user information and assistant replies differently. This approach keeps answers accurate while using much less memory during conversations, making AI assistants faster and better. When tested, Scalaris improved response quality significantly compared to previous systems.
What this means in practice
- •For enterprise software teams: Build personalized AI assistants that efficiently manage user memory for improved customer interactions and response speed.
- •For cloud infrastructure operators: Reduce computational costs and latency in large-scale conversational AI deployments by optimizing memory retrieval processes.
- •For customer support platform vendors: Develop AI-powered support tools that deliver more accurate and context-aware responses with reduced computational expense.$Commercial implications: Enables commercial AI customer support products with scalable, personalized memory improving response quality and efficiency.
Authors
Devarsh Gandhi, Sarthak Pawar, Hemprasad Badgujar, Vaibhav Rathore, Siddharth Sureka, Neha Bhargava
Abstract
Enterprise conversational systems require memory to personalize interactions, retain information across sessions, and maintain conversational coherence. At enterprise scale, however, memory must not only retrieve relevant information accurately but also provide it efficiently to the answer-generation model. Existing memory systems face a trade-off between retrieval accuracy and context efficiency. Retrieving more context can improve recall but increases inference cost and latency. We present Scalaris, a scalable enterprise memory architecture that addresses this trade-off through a unified long-term and short-term memory design. We propose that user interactions and assistant responses possess distinct characteristics and roles in a conversation: user interactions capture evolving knowledge about the user and provide value across sessions, whereas assistant responses must be stored efficiently so that only information relevant to the current query is retrieved. We therefore treat them separately and support each with specialized representations and ingestion/retrieval pipelines. Scalaris reaches 92.2 % accuracy on LongMemEval-S and 90.77 % on LoCoMo using Qwen3-32B, state-of-the-art results on both benchmarks, while supplying the answer-generation model with approximately 10 times fewer retrieved context tokens than Hindsight, a leading memory system for conversational AI. When integrated with our proprietary closed-source AI assistants, Scalaris improved overall response quality by 36 %. Together, these results establish Scalaris as a cost-effective framework for building scalable, personalized conversational AI systems.