Live engines cite mostly different web pages for same prompts
Scoring With the Engine: Retrieval Exposure, Cross-Engine Divergence, and the Limits of Engine-Agnostic GEO Scores
Information Retrieval
Summary
This paper looks at how different popular AI engines like ChatGPT and Google choose web pages they mention as answers. The authors found that when asked the same questions, these engines rarely reference the same exact web pages, showing very little overlap. They point out that measuring page quality without considering which engine actually shows or selects those pages doesn’t capture the full picture. Their work suggests it's important to distinguish between how good a page is, how often engines see it, and how often they pick it to show to users.
What this means in practice
- •For search engine engineers: Assess how different AI search engines expose and select URLs for the same queries to improve citation diversity and relevance.
- •For content platform operators: Measure distinct web page visibility across multiple AI engines to better understand evolving content exposure dynamics.
Authors
Benjamin Tannenbaum
Abstract
Recent work asks whether generative-engine visibility can be approximated with deterministic, engine-free page scores. We separate two stages such scores can conflate: exposure to a live engine and citation selection conditional on exposure. In an observational audit of ChatGPT, Microsoft Copilot, Google, and Perplexity, 15 fixed commercial prompts produced 589 citation observations on 6 June 2026, corresponding to 528 unique URLs and 356 domains. Same-prompt cross-engine URL overlap was extremely small: mean pairwise Jaccard similarity was 0.0079, the median was zero, and 84.9% of engine pairs shared no cited URL. On the ten prompts observed on all four engines, mean exact-URL Jaccard was 0.0072. A matched-size hypergeometric baseline preserving each prompt's four-engine URL universe and each engine's list length predicts 0.1272, so observed overlap was only 5.7% of that baseline; zero URL overlap occurred in 86.7% of comparisons versus 12.3% expected. Top-five exact-URL overlap was zero in all 60 pairwise comparisons. A single engine captured only 11.4%-42.6% of the four-engine URL union, and 96.4% of observed URLs appeared in only one engine. A separate 5-to-6 June same-engine comparison found 67.0% mean URL-set turnover. These results do not invalidate engine-free page scoring; they identify its estimand. A score computed without a live engine can estimate page quality or query-page fit, while end-to-end visibility additionally depends on engine-specific exposure and selection. We therefore argue for reporting page fit, observed exposure, conditional selection, and final visibility as distinct quantities.