Personalized memory management improves language model responses
Learning Personalized Memory Gating from Delayed One-Sided Feedback
Machine Learning
Summary
Language models can remember important facts to make future responses better, but they can’t remember everything. The authors introduce a way to decide which facts to keep based on how likely a specific user will need them later. They also created a method that learns from past data and adjusts on the fly to users’ preferences, even when feedback about fact usage is delayed. Their experiments show this method works better than other approaches across different memory sizes and conversation lengths.
What this means in practice
- •For chatbot developers: Improve chatbot relevance by dynamically storing user-specific facts for future conversation use despite delayed feedback signals.
- •For customer support platform teams: Enhance support agents’ language models to remember and reuse information specific to each customer, improving interaction quality over time.
Authors
Kaixuan Ji, Zixiang Chen, Wenting Zhao, Cheng Qian, Heng Wang, Weizhi Zhang, Jielin Qiu, Liangwei Yang, Quanquan Gu, Silvio Savarese, Shelby Heinecke, Huan Wang
Abstract
Long-term memory enables language-model agents to personalize future interactions, but a finite memory bank cannot preserve every extracted fact. Existing memory gating policies are typically shared across users and therefore fail to account for user-specific patterns of future reuse. In this paper, we study memory gating at the fact level. We first propose a retrieval-based labeling strategy that labels each fact by whether it is later retrieved and incorporated into responses. We then formulate online memory gating as contextual apple tasting with delayed one-sided feedback, and propose Warm-Start UCB, which combines offline learning from existing data with online adaptation to the current user. Our theoretical analysis establishes a high-probability regret upper bound of $\tilde{O}(Tε\wedge d\sqrt{T})$ when the delay window is at most $O(\sqrt{T})$, where $T$ is the total number of rounds, $ε$ is the initialization error, $d$ is the dimension of the feature map, and $\tilde{O}(\cdot)$ hides constant and poly-logarithmic factors. The results of our experiment demonstrate that our method outperforms all baselines under different memory budgets and history lengths, establishing it as a promising personalized memory gating policy.