Language model reasoning requires new tests for reliable self-checking
Realized versus Population Reliability: Anytime-Valid Self-Consistency Certification under Correlated Reasoning
Artificial IntelligenceMachine Learning
Summary
When language models try to reason through problems several times, the usual tests to check their reliability assume that these attempts are independent. The authors show that this assumption is often false because attempts can be related or share common parts. They developed new ways to understand and fix these dependencies so that reliability checks remain trustworthy even when the model’s attempts are related. Their findings show when traditional checks work and when special adjustments are needed to be confident about a model’s reasoning.
What this means in practice
- •For ai system builders: Design reliable stopping rules for language model reasoning when multiple related outputs are generated from the same prompt to ensure valid confidence claims.
- •For data quality teams: Implement improved procedures verifying language model reasoning consistency to reduce false alarms and better assess population-level model trustworthiness.
Authors
Terence D. Agbeyegbe, Toneukarin G. Agbeyegbe
Abstract
Self-consistency certification treats repeated sampling from a language model as sequential evidence accumulation and stops when an anytime-valid e-process crosses a threshold, guaranteeing finite-sample Type I error control by Ville's inequality. Prior constructions assume sampled rollouts are conditionally independent. We show this assumption fails in practice: reasoning streams are positively correlated through shared reasoning prefixes or across-prompt heterogeneity, changing not merely certificate efficiency but the hypothesis certified. Modeling dependence via a per-unit latent state (the random-effects / intraclass-correlation structure) induces exchangeable, equi-correlated increments with correlation $ρ$. Correlation splits the single null into a realized null about the drawn belief and a population null about average model behavior, which coincide only when $ρ=0$. We show: (i) the standard betting e-process remains anytime-valid for the realized null, not the population null; (ii) read as a population-null test, a suitable mixture has asymptotic crossing probability at least $π=P(μ(Z)>0)$, potentially exceeding nominal $ε$; (iii) the within-unit mean's effective sample size saturates at $1/ρ$, so extra within-unit rollouts cannot restore population validity; and (iv) a prompt-level e-process restores population validity, with optimal allocation at one rollout per prompt under a pure rollout budget. Simulations and Claude Haiku 4.5 rollouts confirm this regime dependence: on GSM8K ($\hatρ=0.435$, $\hatθ=+0.96$) the gap is dormant, whereas on AIME ($\hatρ=0.480$, $\hatθ=-0.15$) the single-unit rule crosses on 24% of prompts at nominal $ε=0.05$, while the prompt-level procedure declines (0.00). This correctness result identifies which reliability claim an anytime-valid certificate actually supports under dependence.