FIBER improves detection of AI-generated text across domains

FIBER: Bayesian Fusion of Fast-DetectGPT and IRT-Informed Detector Evidence for LLM-Generated Text Detection

Computation and Language

Summary

It can be hard to spot text written by AI models, especially when the style or topic changes. The authors created FIBER, a method that combines scores from a fast AI text detector with additional signals, using a statistical approach that learns from past detector behavior. This combination helps detect AI-written text more accurately across different topics and settings. They also made a version that works well even without past data.

What this means in practice

  • For content moderation teams: Improve automated systems that flag AI-generated text to adapt across different topics and writing styles more reliably.
  • For digital forensics analysts: Enhance tools to identify AI-written content when dealing with texts from unknown or varied sources without prior labeled data.

Authors

Yan Li, Guangsheng Bao, Zhipeng Chen, Guandong Zhang, Yanbin Zhao

Abstract

Detecting large language model (LLM)-generated text remains challenging under domain and generator shifts. Supervised detectors may generalize poorly, while zero-shot methods often rely on individual statistical signals without leveraging historical detector behavior. We propose FIBER (Fast-DetectGPT with IRT-informed Bayesian Evidence Recalibration), which integrates continuous Fast-DetectGPT scores with auxiliary detector outputs through Bayesian evidence fusion. Item response theory (IRT) models historical detector-correctness patterns, providing class-conditional detector-output likelihoods through marginalization over latent text representations. These likelihoods are combined with Fast-DetectGPT score densities and a historical class prior for prediction. When historical Fast-DetectGPT scores are unavailable, a variant called FIBER-Stream estimates class-specific score distributions from an unlabeled batch. On a 32,000-text multi-domain benchmark, FIBER achieves the highest accuracy (0.620) and improves mean AUC over Fast-DetectGPT (0.664 versus 0.600) under five-fold cross-validation. FIBER also consistently improves AUC across four individual domains. Under out-of-distribution transfer, FIBER-Stream achieved an AUC of 0.911, compared with 0.889 for FIBER and 0.922 for Fast-DetectGPT alone. These results demonstrate the potential of integrating complementary detector evidence and historical information to improve detection performance across heterogeneous settings.