Language model tool spots false claims and cuts errors in specialized fields
Domain-Specific Hallucination Detection in Large Language Models
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Large language models sometimes create false or untrue statements called hallucinations. The authors created a system that uses several techniques to spot these false claims with high accuracy across different types of tasks like questions, summaries, and dialogues. They also showed that training models specifically for certain fields like biomedicine works better than using a general approach. Additionally, they improved a language model to reduce how often it hallucinates. This helps make AI-generated text more trustworthy.
large language modelhallucinationclassificationMonte Carlo dropouttemperature scalingcalibrationentailment reasoningDirect Preference Optimizationdomain adaptationbenchmarks
Authors
Varun Teja Chundru, Debasmita Biswas
Abstract
Large language models generate fluent text that can contain unfaithful claims -- a phenomenon known as hallucination. We present a multi-signal detection pipeline combining fine-tuned DeBERTa-v3 classification, Monte Carlo (MC) Dropout uncertainty quantification, and temperature-scaled calibration for response-level hallucination detection. Evaluated on the HaluEval benchmark, our pipeline achieves F1=0.915 and AUROC=0.977 on general-domain tasks, with per-task F1 scores of 0.97 (QA), 0.96 (Summarization), and 0.82 (Dialogue). MC Dropout inference further improves accuracy to 93.2%. A context ablation study confirms the model performs genuine entailment reasoning rather than exploiting surface patterns, with summarization F1 dropping 24% when knowledge context is removed. Learning curve analysis reveals that 25% of training data captures 77% of full-data performance. Beyond detection, we apply Direct Preference Optimization (DPO) to a Qwen2.5-0.5B generator, reducing its hallucination rate from 85.5% to 37.7% (55.9% relative reduction) as measured by our detector. Cross-domain evaluation on the SciFact biomedical benchmark shows that general-domain training transfers poorly (F1=0.52), motivating domain-specific fine-tuning. PubMedBERT fine-tuned on SciFact achieves F1=0.63 and AUROC=0.81, demonstrating that domain-matched pre-training is the strongest adaptation strategy. Code and models are available at https://github.com/varunteja99/hallucination-detection-nlp