Lazy Grounding: Attacking Search Agents with Factual Evidence
2026-08-31 • Computation and Language
Computation and Language
AI summaryⓘ
The authors explain that search agents, which answer questions by looking up information online, can get tricked not just by false info but also by correct info that is about a similar but different question. They call this problem 'lazy grounding' because the agent picks an answer nearby in meaning instead of the right one. They tested this on many models and found accuracy dropped noticeably when the agents used these nearby but irrelevant facts. The study shows that it’s important to make search agents smart enough to know when facts are about a different question, not just to avoid false info.
search agentshallucinationretrievalpoisoned corporalazy groundingfactual evidencebenchmark questionsaccuracyanswer adoptionmisinformation
Authors
Yulin Zhang, Yukun Huang, Sanxing Chen, Tianyi Lin, Ziang Yang, Xunjian Yin, Bhuwan Dhingra
Abstract
Search agents reduce hallucination by grounding answers in retrieved web evidence. Yet reliance on retrieval also creates an attack surface: poisoned corpora with false or malicious documents can cause agents to reproduce misinformation. We show that falsehood is not necessary -- a search agent can be misled by factual evidence for a nearby question, adopting that nearby answer even when it does not answer the current question. We call this failure lazy grounding. We expose lazy grounding using nearby evidence from answer-changing rewrites of benchmark questions. Each document truthfully supports a neighboring rewritten question, but is surfaced for the original question. Across 12 model-benchmark pairs, nearby evidence reduces accuracy by 5.9 points on average and by up to 17.3 points, while inducing nearby-answer adoption in every setting. The effect is stronger when nearby evidence appears later or is more answer-shaped. Our results show that robust search agents must defend against not only misinformation but also the misapplication of factual evidence. The code is publicly available at https://github.com/frankyzha/lazy-grounding.