Composable memory reduces costs and speeds personalized item retrieval
CRIM: Composable and Reusable Item Memory for Efficient Personalized Retrieval
Information RetrievalComputation and Language
Summary
Personalized search and recommendations become slower and more costly as users interact more, since systems repeatedly analyze the same item information. The authors propose CRIM, which creates small, reusable summaries of items independent of the user or query, so these can be quickly combined for each request. This reduces the need to reprocess item details each time and speeds up responses without losing personalization quality. They also train the system by sometimes re-encoding fresh item info to keep summaries accurate. Tests show CRIM performs well on several datasets while cutting computational costs.
What this means in practice
- •For search engine engineers: Cut inference costs and speed up personalized search by caching item representations across queries.
- •For recommendation system developers: Implement reusable item memories to efficiently update recommendations based on long user histories without redundant encoding.
- •For advertising technology teams: Enhance real-time personalized ad retrieval by using compact item memories for faster query responses.$Commercial implications: Enables faster, cost-effective personalized ad delivery by reducing computational redundancy in real time.
Authors
Haolin Li, Haipeng Zhang, Mang Li, Zhaohui Xia, Xun Zhu, Miao Li, Yu Zhang, Ji Wu
Abstract
Large language models incur substantial training and inference costs in personalized search and recommendation, as growing interaction histories lengthen request inputs and repeatedly expose the same item text to the model. We propose \textbf{C}omposable and \textbf{R}eusable \textbf{I}tem \textbf{M}emory (CRIM), a framework that separates item-level semantic encoding from request-level history modeling. A small set of learnable summary tokens compresses each item into compact continuous memories, while segment-level readout tokens (REC tokens) introduce predictive supervision within the interaction history. These memories are independent of users and queries and can therefore be reused across requests. At inference time, CRIM composes cached memories in interaction order and incorporates an optional query to produce a personalized retrieval representation, reducing repeated item-text encoding while retaining request-dependent contextual modeling. We further introduce a joint training mechanism that combines cached memories with sampled live re-encoding. Re-encoding only a few history items per request allows the retrieval objective to jointly optimize the item and request encoders at limited computational cost, while periodic cache refreshes maintain consistency. Extensive evaluation and cost analysis on MovieLens-1M, Qilin, and our new LazSeq dataset show that CRIM achieves state-of-the-art retrieval performance with substantially lower inference costs at the same backbone scale, while limiting item re-encoding during joint training. We will publicly release our code and dataset.