Generative retrieval improves item suggestions while reducing search costs

PinGER: Token-Efficient Generative Retrieval with Semantic IDs

Information Retrieval

Summary

Finding related items online often relies on searching through many possibilities, which can be slow and costly. This paper presents PinGER, a way to represent items compactly so a computer can generate identifiers for them step-by-step instead of searching through all items directly. The authors show that this method, used at Pinterest, makes the system faster and cheaper while improving how often users save related items. They achieve this by combining efficient input handling, controls over what the model generates, and a smart serving setup.

What this means in practice

  • For recommendation system teams: Deploy PinGER to serve related item recommendations with reduced latency and computational costs by replacing nearest-neighbor searches.
  • For search infrastructure engineers: Implement generative retrieval architectures that generate compact semantic IDs to minimize context window use and speed up item retrieval.

Authors

Xinyi Li, Sujan Reddy Ale, Yi-Ping Hsu, Zelun Wang, Yilin Chen, Hanyu Li, Hanlin Lu, Arkin Dharawat, Lulu Yao, Yaxin Li, Bella Huang, Xiangyi Chen, Jaewon Yang, Jiajing Xu, Charles Rosenberg

Abstract

Generative retrieval replaces nearest-neighbor search with token-by-token decoding of an item's semantic ID (SID). But existing SID-based methods primarily use L levels of tokens sequentially to represent each item in the input history. As a result, a fixed context window holds L times fewer items than it would with one embedding per item. We present PinGER, a generative retriever deployed at Pinterest that retains SID outputs while making input histories compact, built on three ideas. (1) An asymmetric attention mask over interleaved SID and dense tokens lets later items attend only to the dense ones, so the model is supervised at every SID position in training yet serves one token per item while computing the same function. (2) Generation is controllable through goal-conditioned outputs, reinforcement learning, and a prefix-diversity penalty, which together steer retrieval toward surface-specific goals and diverse candidates. (3) A serving stack built for efficient wide-beam generation is paired with decoding constrained to valid identifiers and candidate resolution from a host-local index. Deployed in Related Pins, PinGER lifts saves of fresh Pins by 3.60% and overall saves by 0.62%, while reducing the surface's end-to-end latency by 0.7% and eliminating the cost of the nearest-neighbor search it replaces.