Procedure uncovers specific hallucinations in language model responses
Target-Aware Curation of Hallucination Stress Tests for Large Language Models
Computation and LanguageArtificial Intelligence
Summary
Large language models can sometimes give answers with made-up information called hallucinations. The authors developed a way to change test questions step-by-step to expose these hallucinations in a targeted model. They use an AI editor to rewrite questions based on the model’s previous answers and ratings. Their method makes the hallucinations clearer and helps identify where models fail, even when average scores look similar across different models.
What this means in practice
- •For ai system testers: Create targeted test prompts that reveal specific hallucination weaknesses in language models to improve model evaluation.
- •For ai safety teams: Use curated hallucination stress tests to identify unsupported claims in language model outputs before deployment in sensitive settings.
Authors
Jiho Lee, Bingquan Zhang, Lin Ting Liang, Soroush Vosoughi, Weicheng Ma
Abstract
Hallucination stress tests can yield similar average scores across language models even when different prompts elicit hallucinations in each model. We develop a procedure that iteratively edits source-grounded seed prompts to expose hallucinations in a target model. An LLM editor revises each prompt in response to the target's answer and a judge's category-specific score and explanation. In the main setting, the target also receives feedback on its previous response. Experiments cover 500 seeds from seven datasets and ten topics, with intrinsic and extrinsic hallucinations evaluated across open-weight and closed-source models. Editing substantially increases judge-rated severity over the seeds. In the baseline configuration, retained responses fall in the judge's strong-hallucination bin in 77.2% of runs targeting extrinsic hallucination and 74.2% targeting intrinsic hallucination. Later editing rounds account for more than half of the mean score increase. Among 219 prompts that pass model-assisted source-validity checks, 89 elicit responses containing unsupported claims. Across five models, 200 curated prompts yield similar average severity scores, although pairwise Spearman correlations between prompt-level scores are only $ρ=0.0387$ to $0.1897$. Curated prompts can be reused across models, but their diagnostic value depends on source validity and the failures they expose in each model.