More biomedical evidence can increase false positive claims in research
When More Evidence Hurts: Publication-Bias Drift and Principled Stopping for Biomedical Causal Search
Artificial IntelligenceInformation Retrieval
Summary
When computer systems look through medical studies to find proof that a treatment works, finding more papers can actually make the system more likely to wrongly say a treatment is helpful when it is not. This happens because the published studies tend to show positive results more often than negative or neutral ones. The authors call this problem "evidence drift" and show mathematically that the risk of false positives grows as more studies are checked. They also build a system that carefully decides when to stop looking for more evidence, which reduces these mistakes and makes conclusions more accurate and efficient.
What this means in practice
- •For biomedical ai developers: Improve automated systems that combine medical study results by stopping evidence search when adding more data worsens reliability.
- •For clinical data analysts: Use drift-aware retrieval methods to reduce false claims when summarizing medical evidence from published studies.
Authors
Fred Sun, Shangqi Guo
Abstract
Automated biomedical evidence synthesis depends on retrieving published studies, but the biomedical literature is systematically skewed toward positive findings. Deeper retrieval can therefore make a system \emph{more} likely to falsely infer benefit when the true effect is null. We formalise this phenomenon as \emph{evidence drift} and prove that, under a standard publication-bias model, the false-positive probability on null-effect queries follows a strictly increasing large-sample envelope in retrieval depth, approaching one. Empirically, on a held-out test set of 140 Cochrane-derived queries, drift rises monotonically from 7.9\% to 15.7\% as the retrieval budget grows from 3 to 20 steps, and concentrates in the null-effect class. We present DACG-agent, a drift-aware causal-graph agent that incrementally builds a causal knowledge graph from PubMed abstracts and applies a two-layer stopping policy with complementary roles: a KL-divergence monitor that detects posterior convergence (the accuracy layer), and a Bradley--Terry process reward model (PRM) whose online decline detection halts retrieval once evidence quality peaks (the efficiency layer). Against full-budget retrieval, DACG-agent reduces evidence drift from 15.7\% to 6.4\% and improves null-effect accuracy by 21 percentage points (40.0\%$\to$61.4\%) while using 67\% fewer retrieval steps; overall accuracy rises from 61.4\% to 69.3\% (95\% CI 61--77). A simulation confirms the drift result transfers from the analysed vote-counting aggregator to the deployed noisy-OR one.