AI summaryⓘ
The authors present SEVA, a system designed to verify the accuracy of information generated by large language models (LLMs). Unlike previous verifiers that give only simple yes/no answers, SEVA provides detailed evidence, confidence levels, error types, and suggestions for fixes. Training SEVA with reinforcement learning was challenging because standard reward methods failed, so the authors created a new reward system that breaks down verification quality into multiple parts, helping the model learn better step-by-step. They also found that repeated self-improvement rounds made the model specialize in certain tasks rather than becoming broadly accurate. SEVA performed similarly to a top model but provided more transparent, detailed feedback.
HallucinationLarge Language Models (LLMs)Fact VerificationReinforcement Learning (RL)Reward DecompositionSelf-correctionEvidence AlignmentStep-by-step ReasoningConfidence CalibrationError Diagnosis
Authors
Aojie Yuan, Yi Nian, Haiyue Zhang, Zijian Su, Yue Zhao
Abstract
Hallucination is the reliability bottleneck for LLM-based agents, and fact attribution verifiers are the last line of defense -- yet today's verifiers emit only opaque binary labels, leaving agents unable to self-correct and operators unable to audit. We present SEVA, a structured verification agent that emits evidence alignments, step-by-step reasoning chains, calibrated confidence, and a six-category error diagnosis with actionable fixes. Training such an agent with RL is non-trivial: standard binary reward on multi-component output triggers advantage collapse -- within-group reward variance vanishes and the GRPO gradient disappears. We resolve this with a process reward that decomposes verification quality into five independent components weighted 70/30 toward process signals, restoring the gradient and inducing an implicit curriculum -- the agent first masters verification behavior (alignment 0.917 -> 0.997, format 72% -> 100%), then outcomes (F1 64.9 -> 69.0). Structured output further enables a Verify -> Reflect -> Probe -> Refine self-evolution loop, which over four rounds on a 7B model surfaces an unexpected structural finding: each round produces a benchmark-specialist, not a generalist (+15 pp on HaluEval, -10 to -14 pp on TruthfulQA in the same model, persistent at 4x data). On ClearFacts, SEVA-3B matches GPT-4o-mini (69.0 vs. 69.8 F1) while producing substantially richer, auditable output -- confirming a principle that should generalize: for any RL task with multi-component generation, reward granularity must match output granularity.