A Dual-Hypothesis Reasoning Framework for LLM Guardrails
2026-07-20 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors introduce ARBITER, a new system to help large language models (LLMs) decide if a prompt is safe or not by thinking about both safe and unsafe meanings before choosing. They also propose a special way to train the model that breaks down its output into parts and focuses more on important parts. Unlike other methods that are costly and complex, ARBITER uses cheaper techniques to train and still performs better. It also explains why it thinks something is unsafe, making it easier to understand. Tests on safety tasks show ARBITER works better than current methods, especially on new kinds of inputs.
Large Language ModelsSafety ModerationDual-Hypothesis ReasoningSupervised Fine-TuningLoRA Fine-TuningReasoning TracesOut-of-Domain EvaluationModel InterpretabilityGuardrails in AI
Authors
Md Asiful Islam, Mihai Surdeanu
Abstract
We propose ARBITER, a novel LLM guardrail framework that introduces two key ideas: (i) dual-hypothesis reasoning, a reasoning method for LLM guardrails that explicitly considers both safe and unsafe interpretations of a prompt before making a safety decision, and (ii) multi-component supervised fine-tuning (MC-SFT), a structured training loss for reasoning-based guardrails that decomposes LLM outputs into logical components and weights them according to their importance. Existing reasoning-based guardrails often rely on expensive procedures, such as generating reasoning traces using larger or closed-source teacher models and applying full-parameter fine-tuning. In contrast, ARBITER uses a cost-effective self-generation strategy for reasoning traces and LoRA-based parameter-efficient fine-tuning while still achieving better performance than these expensive approaches. Additionally, ARBITER provides faithful evidence-phrase explanations for unsafe decisions, enabling a more transparent and interpretable guardrail method. Experiments on three safety moderation benchmarks show that ARBITER outperforms existing reasoning-based and non-reasoning guardrail baselines, with clear gains in out-of-domain evaluations.