LLM research agents often make stronger claims than documents support
Quantifying Claim Smoothing by LLM Research Agents
Computation and Language
Summary
When large language models (LLMs) read many documents to write research reports, they tend to make conclusions that sound more confident than the actual evidence allows. This happens because the models overlook differences and disagreements in the original sources and create a smoother, more decisive story. The authors call this problem claim smoothing. They built a system named SmoothScore to measure how much the models smooth claims by comparing claims to their supporting evidence. They tested this on several systems and datasets and found that claim smoothing is a common issue.
What this means in practice
- •For ai developers: Evaluate and improve the reliability of research summaries produced by LLM-based agents by identifying overstated claims.
- •For legal analysts: Detect overconfidence in AI-generated reports that synthesize large document collections, helping to assess claim validity.
Authors
Tiasa Singha Roy, Manya Wadhwa, Greg Durrett
Abstract
Deep research, or any use of large language models (LLMs) to read a large document collection and produce insights, requires synthesizing information from different sources. In this work, we find that LLMs produce long-form research reports that read as confident syntheses but which fail to accurately reflect the underlying documents. We call this failure smoothing. In collapsing diverse and potentially conflicting sources into a single narrative, the model discards the evidential "jaggedness" of its corpus and asserts conclusions more decisive than the citations can support. We introduce a claim level scoring pipeline, SmoothScore, to quantify this effect. Our pipeline extracts key conclusive claims from model generated reports, then finds evidence relevant to each claim and clusters that evidence along high level aspects. Finally, each claim is scored with respect to the evidence to quantify the degree of smoothing done by the agent. Across 7 systems and 4 datasets, we find that agents consistently produce stronger claims than warranted.