Iterate Until Retrieved: Factual Nugget Optimization for Discoverable Continual Corrections in Agentic RAG
2026-05-25 • Computation and Language
Computation and Language
AI summaryⓘ
The authors study a system that helps businesses answer questions using stored knowledge, focusing on when users give specific factual corrections instead of general feedback. They create small, clear knowledge entries called factual nuggets from these corrections. Their method, Iterative Nugget Optimization (INO), improves these nuggets step-by-step to make sure the system can find and use them correctly. They tested this method with real business agents and found it made the system better at using the corrected facts.
Retrieval-Augmented GenerationAgentic SystemsFactual CorrectionsKnowledge BaseIterative OptimizationBusiness-to-Business (B2B)Support AgentsInformation RetrievalNatural Language ProcessingAutomated Evaluation
Authors
Moshe Hazoom, Gal Patel, Alon Talmor, Tom Hope
Abstract
Agentic retrieval-augmented generation (RAG) systems in complex B2B (business-to-business) settings may often receive free-form response feedback. Rather than generic feedback signals such as style, preference, or overall response quality, we focus on actionable factual corrections. We identify these instances and convert them into compact knowledge-base entries, which we call factual nuggets. We introduce Iterative Nugget Optimization (INO), an index-time optimization method that uses the production agentic RAG as a test harness: it creates an initial nugget, probes it with the triggering query and paraphrases, reflects over failed retrieval and answer traces, and revises the nugget until it is discoverable. We evaluate INO with two production B2B knowledge-assistance agents across multiple companies that use our system: a product support agent that answers questions over company-specific knowledge bases, and a support ticket agent that assists support engineers. INO consistently improves results over baselines in terms of discoverability and usage of factual corrections, in automated and human evaluations.