Deep research system shows low self-consistency in entity and citation outputs
Self-Consistency of Entity Sets and Citations in a Deep Research System under Repeated Prompting
Computation and Language
Summary
When asked the same question multiple times, a popular AI research tool gave very different answers each time about cybersecurity topics. The differences showed up in the specific names and citations it mentioned, which rarely overlapped across tries. The authors found that these inconsistencies were tied to the type of information rather than just random changes in the system. This means the tool's answers can vary a lot even if you don’t change the question.
What this means in practice
- •For data engineers: Evaluate the stability of extracted entities and citations from repeated AI queries to improve data quality in knowledge management systems.
- •For cybersecurity analysts: Assess reliability of AI-generated research reports by comparing consistency of entity mentions and references over multiple runs.
Tested on one dataset.
Authors
Lauri Pykälä, Kimmo K. Kaski
Abstract
Deep research agents (DRA) are commonly used for open-ended synthesis tasks where closed-world QA benchmarks are poorly suited to characterize relevant aspects of performance. We study the self-consistency of one commercial deep research system under repeated identical prompting, representing each report with the set of named entities it mentions, categorised by a custom schema, and the set of citations it retrieves. Across repeated runs of a single open-ended cybersecurity related prompt, we quantify pairwise overlap to characterize how stable the retrieved and synthesized output structure is under repeated identical prompts. In ten runs, the pairwise entity Jaccard results in a median of 0.12 and the pairwise citation Jaccard is below 0.02, with discovery curves that do not visibly saturate. The magnitude and structure of (dis)agreement between entity sets varies by entity class, which suggests that self-consistency in a DRA is better understood as a function of the underlying entity space than as a stable property of the system.