Guardian Crawler: Retrieval-First Knowledge Discovery with Bounded LLM Augmentation for Noisy Web Intelligence
2026-08-10 • Information Retrieval
Information RetrievalArtificial IntelligenceComputation and LanguageMachine Learning
AI summaryⓘ
The authors created a system called Guardian Crawler to help find useful information from messy, noisy web-like data, especially in tricky areas with incomplete or mixed reports. Their approach uses a mix of search and smart ranking to pick the best documents before generating summaries that clearly cite sources. Tests on fake web data showed their method retrieves more relevant results than a basic search method. Although the results are promising, the authors note that more work is needed to prove how well it works in real-world situations and to confirm summary accuracy with humans.
BM25 retrievalembedding-augmented rerankingknowledge discoveryevidence-grounded summarizationconstrained generationNDCG@10P@10synthetic web corpusrisk-aware rankingdocument citation
Authors
Joshua Castillo, Santosh Nukavarapu, Ravi Mukkamala
Abstract
Retrieving relevant evidence from noisy web data is challenging, particularly in sensitive domains containing incomplete reports, heterogeneous language, and irrelevant content. We present Guardian Crawler, a reproducible retrieval-first testbed for controlled experiments on knowledge discovery and evidence-grounded summarization over synthetic web-like corpora. The architecture combines BM25 retrieval with risk-aware, embedding-augmented, and hybrid reranking, followed by constrained retrieval-augmented generation with explicit document citations. Experiments on a synthetic 900-document corpus and 10 queries produced the highest descriptive retrieval scores under risk-based reranking, with P@10 = 1.00 and NDCG@10 = 0.94, compared with 0.94 and 0.81 for BM25. The best hybrid and BM25+Semantic configurations reached NDCG@10 values of 0.94 and 0.88, respectively. All 41 evaluable generated bullets passed the lexical coverage threshold; an automated LLM judge classified 36 as supported, one as partially supported, and four as unsupported. These results demonstrate the feasibility of Guardian Crawler as a controlled testbed but do not establish statistical superiority, human-validated faithfulness, or transfer to live-web investigative environments.