LENS: In-Context Search via Latent Evidence Exploration over Dynamic Raw Documents
2026-08-17 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors present LENS, a method to help language models find the right parts of changing documents to answer questions without building fixed indexes first. Instead of splitting documents ahead of time, LENS guesses which parts might help answer a question and checks them step-by-step, using the model to decide where to look next. This approach adapts quickly to document updates and better matches answers to supporting evidence than some older methods. Their tests show LENS finds more relevant evidence and keeps answer quality high, even without pre-made indexes or preprocessing.
LLM agentsretrieval-augmented methodsevidence localizationindex-free searchdynamic document collectionsquery-conditioned beliefself-organizing knowledge clustersexact matchReAct baselineclosed-book QA
Authors
Xingjun Wang, Gongsheng Li, Qi Fan, Yunlin Mao, Luyan Su, Yingda Chen
Abstract
LLM agents increasingly answer questions over dynamic raw-document collections, where files may change before preprocessing, and relevant evidence (spans, sections, pages, or tables) is query-dependent. Existing retrieval-augmented approaches pre-materialize evidence via fixed chunking, embeddings, or persistent indexes: effective for lookup, yet costly, stale-prone, and committed to a granularity before the query is known. We formulate in-context search as Budgeted Evidence Localization over a latent evidence space induced by dynamic raw documents and propose LENS (Latent Evidence Exploration and Search), an index-free framework. Instead of pre-materializing the evidence space, LENS maintains a query-conditioned belief over candidate units, iteratively selecting candidates via complementary lexical, local, and exploratory proposal policies, updating the belief via an LLM relevance oracle, and narrowing toward high-posterior regions under a controllable budget. Evidence is consolidated into compact, source-grounded regions of interest and compressed into self-organizing knowledge clusters reused across related queries. On a controlled 500-question evaluation with matched corpus snapshots, LENS reaches 62.4% exact match and 84.8% evidence recall vs. 65.2% exact match but 50.4% evidence recall for a ReAct-style baseline. Across scales, LENS gives the strongest supporting-fact localization and answer grounding. On a fixed 150-question fullwiki subset over the raw Wikipedia dump with zero indexing, LENS and ReAct are nearly tied in official answer quality (43.3% vs. 42.7% EM), with LENS grounding more answers in retrieved evidence (84.0% vs. 70.7%). A no-retrieval Closed-Book reference highlights the contribution of model memory. LENS is query-ready after corpus changes, needs no preprocessing or persistent index, and preserves source-grounded evidence localization throughout.