System reduces agent memory size while keeping key information intact
LAM: Efficient Lossy Agent Memory Framework With A Retrieval-Score Error Bound
Artificial Intelligence
Summary
Agents that think and act on lots of information need to remember many details, which can slow them down and exceed their memory limits. The authors created LAM, a system that smartly removes repeated or unnecessary details while keeping almost all important information. This approach speeds up the agent’s thinking process without losing the ability to find the best past memories. They also made a way to predict how much faster the system will be before using it.
What this means in practice
- •For ai system engineers: Build agents that handle longer histories with less memory and faster reasoning by removing redundant data before processing.
- •For conversational ai developers: Improve chatbot responsiveness by compacting conversation history while keeping essential information for accurate answers.
Authors
Baixi Sun, Le Chen, Anjir Ahmed Chowdhury, Xiaolong Ma, Chih-Hsuan Yang, Mingze Xia, Syed Zawad, Sheng Di, Rajkumar Kettimuthu, Huihuo Zheng, Rajeev Thakur, Venkatram Vishwanath, Feng Yan
Abstract
Agent memory grows as agents read inputs, reason, and call tools. Longer histories increase inference cost and eventually exceed the context window. LLM-based summarization reduces this history but adds latency and provides no explicit bound on information loss. We propose LAM, a Lossy Agent Memory system with three components: a deterministic deduplication rule with a substitution bound on retrieval scores - a bound on score perturbation, not a certificate of unchanged ranking; a memory manager that preserves the cached prefix and overlaps compaction with inference; and a performance model that estimates compaction costs before deployment. On 600 agent trajectories, LAM removes 22.47% of observation tokens while retaining 99.984% of the measured gold-patch evidence. At a fixed deletion set, the performance model predicts a 71.4x-91.6x end-to-end speedup from removing records before prefill instead of deleting them from a prefilled context. That benefit comes from the schedule rather than the rule and applies to any prefix-preserving test.