ConformalGuard controls false positives for language model actions

ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers

Machine Learning

Summary

Making decisions using language models can sometimes wrongly flag safe inputs as risky, causing unnecessary actions. The paper presents ConformalGuard, a method that wraps around existing language model classifiers to better control how often these false warnings happen, even with limited data. The authors show this method can reduce false alarms while still catching most true problems across different language models. This helps make automated decisions more reliable for tasks like email screening and content moderation.

What this means in practice

  • For email security teams: Improve phishing detection by reducing false alarms while maintaining high threat detection rates using conformal calibration.
  • For content moderation teams: Control the rate of unnecessary content review interventions by applying calibrated action gating to language model classifiers.

Authors

Bishnu Bista, Dhanraj Haibatrao Jadhav, Vibha Choudhary, Gaurav Saxena, Arun Pandiyan Perumal, Arjun Subedi, Jignesh Ammineni

Abstract

Language model classifiers are increasingly deployed for high-stakes decisions such as phishing screening, content moderation, and fraud triage, yet validation-set thresholds provide no finite-sample control of deployment-time false positives. We present ConformalGuard, a model-agnostic calibration layer that wraps a fixed scoring function with finite-sample control over the rate at which legitimate inputs trigger an intervention tier. Our contribution is empirical and operational, not a new conformal theorem: zero-shot LM API scores can be coarse enough (10--11 unique values here) to make temperature scaling and isotonic regression operationally identical to raw thresholding for low-FPR action gating. We compare these methods with conformal calibration for fine-tuned BERT, Claude Sonnet 4, and GPT-4o on public email corpora. For Claude, conformal calibration reduces FPR from 0.040 to 0.008 while recall falls from 0.938 to 0.875. For deterministic GPT-4o scores, the 1% operating point is too conservative to deploy (recall 0.564); three-run stochastic averaging raises recall to 0.732 at FPR 0.005. At 2% prevalence with BERT, conformal calibration removes 25 false-positive reviews per 10,000 emails (95% CI [10, 43]). The guarantee is legitimate-event-rate control under exchangeability; discrimination, workload effects, and robustness to shift remain empirical.