Recommendation feedback guides agent memory to improve future content
How Can Recommendation Feedback Evolve Agent Memory?
Artificial Intelligence
Summary
Recommendation systems give feedback like clicks and conversions to content-generating agents, but these signals are often delayed and mixed up with other factors, making it hard for agents to learn what worked. The authors introduce TIDE, a method that treats stored memories as a changing group that evolves based on delayed feedback. This approach helps agents improve their future decisions by better understanding which past experiences were useful. Tests show it improves important metrics like click rates in real e-commerce scenarios.
What this means in practice
- •For digital marketing teams: Improve personalized promotional content by evolving memory representations based on delayed customer engagement feedback.
- •For e-commerce platform engineers: Enhance recommendation-based content generation agents with a memory evolution system that increases user click and activation rates.$Commercial implications: Enables selling improved content recommendation engines that boost user engagement and conversion rates.
Authors
Shanwen Mao, Mingming Li, Hao Zhang, Zhiheng Li, Yige Wang, Penghua Yu, Junxiong Zhu
Abstract
Content-generation agents continuously receive impressions, clicks, conversions, and negative feedback from recommendation systems, providing real-world outcome signals for memory evolution. However, these signals are delayed and noisy, confounded by audience composition, placement, and recommendation policies, and may result from the combined influence of multiple memories, making accurate attribution difficult. Existing methods rely primarily on immediate feedback or semantic retrieval and therefore struggle to reliably translate recommendation outcomes into memory fitness. To address this challenge, we propose TIDE (Trajectory-Informed Directed Memory Evolution), an external memory evolution framework driven by delayed recommendation feedback. We further introduce Memory Evolution Gain (MEG), which measures the utility improvement of evolved memory over a no memory baseline on strictly future tasks. TIDE treats memory as a capacity-constrained population of experiences: temporal and semantic credit assignment estimates contextual fitness, while responsibility credit distributes outcome signals according to the memories referenced during generation. These signals are then used to reinforce, crossover, mutate, or evict memories. On an e-commerce membership marketing content-generation agent, TIDE achieves a +7.75-percentage-point MEG in offline temporal replay and significantly improves both unique click-through rate (UCTR) and activation rate in an online A/B test. On a delayed-label benchmark, TIDE achieves the lowest mean absolute error (MAE) and root mean squared error (RMSE) and the highest MEG among the compared methods, demonstrating its effectiveness.