MemFit cuts memory costs and speeds up conversation agents
MemFit: Efficient Long-Term Agentic Memory
Artificial Intelligence
Summary
Current large language models remember past conversations by summarizing or reorganizing them, but this can be slow and costly. The authors propose MemFit, which stores conversation turns exactly as they happen, making write operations almost instant and avoiding the need for repeated large model calls. MemFit then finds relevant memories using a method that does not require a heavy model at every step, combining simple keyword and meaning searches with a smarter reranking process. Tests show MemFit remembers better and faster than existing methods, making it easier to build conversational agents with long memory.
What this means in practice
- •For chatbot developers: Implement persistent memory in chatbots with rapid memory storage and efficient retrieval, improving conversation consistency over long interactions.
- •For multimodal ai system builders: Build AI systems that integrate text and visual memories effectively, allowing agents to recall and use information from both modalities without expensive processing.
Authors
Mitchell Piehl, Muchao Ye
Abstract
Long-term memory systems for large language models (LLMs) have gained popularity for extending reasoning capabilities across applications. Current memory systems rely on LLM agents to organize and consolidate memory, resulting in costly, inefficient write operations. To address this limitation, we propose MemFit, a long-term memory system for conversational agents that reduces the cost and latency of memory operations. Unlike existing systems that rely on expensive LLM calls for memory construction or discard surface-level details through compression, MemFit stores each turn verbatim in an append-only store with near-instantaneous, LLM-free insertion, indexing turns with segment summaries rather than replacing them. Additionally, MemFit uses an LLM-free, multi-path retrieval strategy that combines lexical and semantic signals with cross-encoder reranking over caption- augmented episodes in both textual and multimodal settings. Empirical results on three widely used benchmarks, LoCoMo, MemGallery, and LongMemEval-S, show that MemFit achieves state-of-the-art performance while reducing memory construction time and cost several-fold, providing a scalable and efficient solution for persistent agentic memory.