Searchinject reduces unwanted content exposure in language model searches
SearchInject: Retrieval Exposure Control for Indirect Prompt Injection in Language-Model Search
Artificial IntelligenceComputers and SocietyInformation RetrievalMachine Learning
Summary
When a language model uses web pages to answer questions, sometimes unwanted or unsafe content sneaks in indirectly through those pages. The authors propose SearchInject, a method that decides which pages to include by combining several signals, such as relevance and risk of abuse. Their tests show SearchInject greatly lowers exposure to risky content while barely affecting the quality of search results. This approach helps keep language model outputs cleaner without losing useful information.
What this means in practice
- •For search engine developers: Control the admission of retrieved documents to reduce indirect prompt injections in search-augmented language models while maintaining relevance.
- •For content moderation teams: Improve filtering strategies to quarantine potentially harmful or manipulative content before it influences language model responses.
Authors
Kainan Zhou, Gangzhen Qian, Janet Sung, Zhi Jing, Rong Bao, Qianru Lao
Abstract
An indirect prompt injection can affect a search-augmented large language model only after its carrier page reaches model context. SearchInject controls this admission decision with a joint score combining retrieval relevance, search-abuse cues, instruction likelihood, and two relevance interactions. A quarantine threshold is selected under an empirical 0.5% clean-candidate false-quarantine constraint on a calibration split. Evaluation uses 300 held-out SciFact queries and held-out payload categories within four construction families already represented during development. Across 1,200 query-family outcomes per retriever, top-10 target exposure falls from 0.9408 to 0.1583 for BM25, from 0.9158 to 0.1508 for a normalized hybrid, and from 0.8450 to 0.1375 for latent semantic analysis. These are relative reductions of 83.2-83.7%; the largest absolute change in clean normalized discounted cumulative gain at rank 10 is 0.0018. The frozen threshold yields clean test false-quarantine rates of 0.59-0.65%, above the empirical calibration constraint. A query-level analytical reanalysis accounts for the repeated construction outcomes under an independent-query assumption. A separate 104-case generation follow-up records context removal but no confirmed undefended hijacks under a post hoc endpoint, so it cannot estimate a reduction in attack success. The evidence supports retrieval exposure control within the evaluated construction families; the benefit of the interaction terms at matched clean cost remains unresolved.