Interactive memory lets ai improve long conversations over time
Interactive Memory Learning for Long-Term Conversations
Artificial IntelligenceComputation and Language
Summary
Long conversations with AI often struggle because they store too much information without deciding what's really important. The authors present a system called ICML that lets AI learn which parts of a conversation to remember and when to bring that information back up. It uses two AI agents working together—one that decides what to save and another that decides when to retrieve it—and they learn by getting feedback on how well the conversation goes. This approach helps the AI get better at holding conversations the more interactions it has.
What this means in practice
- •For chatbot developers: Create conversational agents that adapt their memory use to improve long-term interaction quality based on ongoing feedback.
- •For customer support teams: Implement AI systems that remember and retrieve relevant prior customer details to enhance personalized service over multiple sessions.
Authors
Cai Ke, Jiangyue Yan, Han Zhang, Xin Liu, Zike Yuan, Yue Yu, Hui Wang, Ruifeng Xu
Abstract
Recent advancements in large language models have significantly enhanced the capabilities of agents in modeling long-term conversations. Despite these successes, existing approaches typically adopt a static heuristic paradigm, where information is passively archived without adaptive memory valuation. Consequently, these methods fail to self-evolve or align their memory management with evolving user needs. To address this, we propose ICML (InteraCtive Memory Learning), a multi-agent framework that transforms the memory mechanism from a passive archive into a learnable, interactive memory policy. Specifically, we first employ a session synthesis pipeline to generate expert data, facilitating rapid test-time adaptation in unseen scenarios. Building on this, ICML utilizes an online reinforcement learning mechanism where a Planner agent selectively encodes high-value information and a Trigger agent dynamically retrieves it to optimize response quality, whereby the two agents co-evolve through continuous interaction feedback. Crucially, both agents are synchronized through a delayed reward mechanism that propagates future feedback back to earlier storage decisions, ensuring memory policies are precisely aligned with user expectations. Experimental results demonstrate that ICML significantly outperforms strong baselines, exhibiting the unique capability to continuously improve response quality as interactions accumulate.