Recommendation agents improve using screenshots and dynamic memory
ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents
Artificial IntelligenceInformation Retrieval
Summary
Recommendation agents help suggest items users might like, but existing ones have trouble understanding messy web pages and remembering long user histories efficiently. The authors propose ReMem, a new system that reads item information from screenshots using OCR, making it less dependent on web page structure and more adaptable. ReMem also uses a special memory method to keep track of user preferences over long times without slowing down. Tests show ReMem performs better than current top methods in recommending, ranking, and judging items.
What this means in practice
- •For e-commerce platform developers: Develop smarter recommendation systems that interpret product pages from images and remember long user histories efficiently.$Commercial implications: Enables building commercial recommendation features that adapt better to diverse online shops by reading visual page data and handling long user interactions.
- •For mobile app developers: Build recommendation agents in apps that process product screenshots and user preferences over time without heavy memory use.
Authors
Haohao Qu, Yongcheng Jing, Chun Hin Chan, Shanru Lin, Wenqi Fan, Dacheng Tao
Abstract
Recent Recommendation Agents (RecAgents) offer a promising alternative by shifting recommendation to an active, user-side paradigm, where generative agents autonomously perceive external platforms, reason over user preferences, and execute decisions. However, existing RecAgents still suffer from two critical limitations: brittle item perception based on noisy and heterogeneous item pages, and inefficient long-context reasoning over extended user histories and multi-step interaction traces. To address these challenges, we propose a novel recommendation agent framework, termed as ReMem, that combines OCR-based multimodal perception with time-evolving dynamic memory. Instead of parsing raw HTML, ReMem observes item pages through screenshots and extracts structured multimodal information via an OCR tool, enabling a more humanoid and platform-agnostic perception mechanism. To support long-horizon preference modeling, ReMem further introduces a chunk-wise sequential memory update strategy, where the agent selectively maintains a fixed-size memory of informative historical interactions while processing arbitrarily long contexts with linear inference complexity and bounded context length. This design allows the agent to preserve evolving user preferences without relying on external memory modules or disrupting the standard autoregressive generation process. To enhance the dynamic memory instruction, we further develop a multi-memory GRPO variant, which propagates the final-answer advantage to all intermediate conversations that contribute to the final response. Extensive experiments on three datasets demonstrate that ReMem consistently outperforms state-of-the-art baselines, achieving an average improvement of 5.16\% across three recommendation agent tasks, namely searching, ranking, and judging.