LLM systems improve approval tracking and error recovery in published answers
Approval Integrity and Recovery in LLM Answer Publication
Cryptography and Security
Summary
Keeping published answers from AI models accurate and trustworthy is challenging because approved content must stay linked to valid approval checks over time. The authors study a system that binds approved content exactly to its authorization context and recovers from errors by rechecking answers along a timeline. Their tests show the approach retains most correct answers and detects problematic ones better than baseline methods, though some errors increase during recovery. They also analyze how freshness and prompt attacks affect integrity, helping separate kinds of approval mistakes.
What this means in practice
- •For ai platform engineers: Enhance AI answer publication systems to better verify and maintain approved content through error recovery and freshness checks.
- •For content moderation teams: Use precise approval tracking to separate stale and false approvals from legitimate responses in automated content pipelines.
Authors
Faruk Alpay, Taylan Alpay
Abstract
Publication integrity in LLM systems requires binding approved content to its current authorization context. We examine exact-content binding, authorization freshness and checkpoint recovery in Lightcap's publication enforcement mechanism. On 900 independently human-annotated RAGTruth responses from 150 source tasks, three dated Ministral models and a same-model direct-grounding baseline yield 3,600 assessments. The production response-act checker instantiated with 14B accepts 291 of 302 unsupported-labelled answers; the direct baseline accepts 41. Supported-answer retention is 95.2% and 66.9%, respectively. An exact promotion-correction identity tracks error through 100 chronological 3B-14B-8B-14B answer trajectories. Among 65 initially approved answers, the final stateful recheck-recovery policy increases exact-match error by 9.23 percentage points relative to the initial checkpoint (95% article-clustered interval [-1.72, 19.61]). Controlled evidence-fingerprint changes expose asymmetric freshness enforcement between publication and recovery. A separate BIPIA prompt-injection experiment records zero target insertions among 266 valid editor outputs. External Hugging Face calibration experiments transfer retrieval models from ArguAna to SciFact and NFCorpus, and diagnostic decision rules from Thunderbird to BGL, distinguishing probability calibration from ranking changes. The measurements separate semantic false approval, stale authorization and recovery-induced error at executable publication boundaries.