From Extraction to Navigation: Progressive Retrieval with Indirectly Infinite Depth
2026-06-29 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors address problems in recommendation systems where usual methods either get stuck in limited user interests or lose track of what the user really wants over time. They introduce IID-Nav, a new approach that actively guides the search for items using a smart policy that adapts to user goals. Their method reuses information across searches to explore deeply without slowing down, and uses special training techniques to keep the search stable and precise. Tests on large datasets show that IID-Nav works better than current methods while being fast. This approach helps recommenders find better matches by navigating item relationships more effectively.
recommender retrievalgraph traversalinterest tunnelsearch driftnavigation policystate evolutionhard negative samplingbillion-level datasetsitem-to-item recommendationlatent inference
Authors
Linxiao Che, Shanshan Huang, Haitao Lu, Yijia Sun, Qiang Luo, Ruiming Tang, Han Li, Kun Gai, Guorui Zhou
Abstract
Modern large-scale recommender retrieval is shifting from static similarity matching to dynamic item space navigation, framing retrieval as iterative goal-driven graph traversal. Conventional item-to-item (i2i) methods fall into the "interest tunnel" and fail to excavate deep user interests, while existing index-based retrieval suffers from persistent "search drift", caused by static entry nodes and fixed graph topologies unable to track shifting real-time user intent. To resolve the above defects, we present IID-Nav, a framework modeling retrieval as stateful autonomous graph exploration with three core contributions: (1) A goal-aware navigation policy substituting passive neighborhood expansion with active intent routing supervised by a target discriminator; (2) A recursive state evolution mechanism supporting Indirectly Infinite Depth (IID) via cross-request state reuse, which enables logical unlimited-depth graph traversal without linearly rising inference latency; (3) A trajectory-aligned training paradigm equipped with graph hard negative sampling to stabilize optimization over full navigation paths. Evaluations on billion-level industrial datasets show IID-Nav surpasses mainstream retrieval baselines under strict latency budgets. Empirical results verify that our method alleviates search drift remarkably and retains high precision for deep retrieval paths, offering an efficient, robust retrieval solution for industrial recommendation systems.