Structured document display shifts citation credit in search results
CITECHOICE: A Causal Audit of How Document Presentation Redistributes Citation Credit in Agentic Search
Computation and Language
Summary
When search engines show multiple documents supporting the same fact, they only cite some of them. The authors studied how changing the way a source is presented affects whether it gets cited. They found that using a structured display boosts the number of times that source is cited, without increasing overall citations or harming others. The order of documents in search results has an even bigger effect on citations than presentation style. The study shows that citation credit can be shifted by how information is shown, but it doesn’t change which sources get included overall.
What this means in practice
- •For search engine designers: Improve algorithms to highlight particular sources by changing document presentation without altering ranking order.
- •For web content strategists: Adjust how factual content is formatted to indirectly increase visible citation credit in search answers.
Authors
Sriram Selvam, Anneswa Ghosh
Abstract
When several retrieved sources support the same claim, an answer engine cites some but not others. We call this decision citation allocation and introduce CITECHOICE, a causal audit of authentic multi-turn agentic search. From 129 everyday-query transcripts, CITECHOICE selects 113 same-call document pairs with independently verified support for the same pre-specified fact, without observing ranks or answer outcomes; blinded human review confirms 103. It runs a hash-verified 2-by-2 replay crossing pair order with jointly generated, fidelity-checked structured and prose renderings of one target while the rest of the transcript remains fixed. Three results emerge. First, and most importantly, structured rendering concentrates citation credit rather than clearly increasing source admission. It raises target citation count by +0.50 citations per answer (95 percent CI [+0.20, +0.84]; Holm-adjusted p=.033), without increasing total citations or reducing competitor credit. The pre-specified incidence effect (whether the target is cited at all) is +4.5 percentage points and inconclusive (95 percent CI [-1.4, +10.4]; p=.168). Second, observational position differences exceed controlled reordering effects: the citation-rate gap between rank 1 and rank 5 is 42.3 percentage points, compared with +7.9 percentage points in the main replay and 0.0 percentage points held out. Third, citation evaluation has a measurable noise floor. Although the aggregate count effect repeats under fresh decoding of 30 frozen families, 15 percent of binary decisions change and decoding accounts for an estimated 45 percent of single-generation family-effect variance. Together, these findings isolate what survives control: presentation can causally redistribute visible citation credit within frozen transcripts. They do not establish reliable source admission, a pure formatting mechanism, or a general rank advantage.