Agents learn what memories to keep under limited resources
Remembering Has a Price: Learning Memory Worth under Resource Constraints
Artificial Intelligence
Summary
Lifelong learning agents need to remember past experiences to improve over time, but when they have very limited memory and processing power, just remembering anything relevant isn't enough. The authors found that choosing which experiences to keep and reuse is crucial because wrong or ineffective memories can waste precious resources. They introduce TokenWorth, a method that learns online which memories are truly valuable based on sparse feedback, improving task success while using less computing power. This approach shows that smarter memory decisions beat simply storing more data when resources are tight.
What this means in practice
- •For ai system developers: Improve AI agents working with limited compute and memory by enabling selective memory retention for better task success.
- •For mobile ai app developers: Create more efficient mobile apps that adapt over time by remembering only the most useful user interactions under memory constraints.
Authors
Guannan Lai, Zi-Yang Bai, Jun-Peng Jiang, Han-Jia Ye
Abstract
Lifelong agents continually accumulate and reuse experience to improve after deployment, yet in practice they often operate under tight generation, interaction, and memory budgets. Under such stringent joint constraints, we find that existing lifelong-memory methods can become ineffective, with several sophisticated approaches offering little advantage over simple retrieval or even no-memory agents. This reveals a more fundamental problem: resource scarcity makes each memory decision more consequential. When only a small amount of past experience can be retained or reused and opportunities for reasoning and recovery are limited, retrieving a relevant but ineffective memory can consume scarce context and hinder downstream execution. The key challenge is therefore not simply to retrieve relevant experience, but to determine which experiences are actually worth retaining and reusing. To address this challenge, we propose TokenWorth, a lightweight online memory framework that learns query-dependent memory worth from sparse execution feedback and applies it consistently to memory retrieval and bounded memory management. Across three LifelongAgentBench environments and both local and API-based deployments, TokenWorth achieves the highest task success rate while using the fewest output tokens among the evaluated methods. These results suggest that effective lifelong adaptation under resource constraints depends not on remembering more, but on learning what is worth remembering. Our code is publicly available at https://github.com/AIGNLAI/TokenWorth.