Truthfulness benchmarks cleaned to avoid misleading answer clues
Judging by the Cover: Cleaning LLM Truthfulness Benchmarks to Avoid Surface-Level Feature Leakage
Computation and LanguageMachine Learning
Summary
Sometimes, tests that check if AI can pick the true answer instead rely on easy clues like word patterns, not real reasoning. The authors found that some AI truthfulness tests had these clues, letting AI do better without truly understanding. They created a way to clean these tests by removing pairs of answers that leak these clues. They also offer a tool so new tests can be cleaned before use to get more honest results.
What this means in practice
- •For ai evaluation teams: Provide cleaner truthfulness test sets that reduce misleading answer biases for more accurate AI assessment.
- •For dataset curators: Use Audit-Prune to identify and remove bias-inducing answer pairs before releasing AI benchmark datasets.
Authors
Foad Namjoo, Remy Ogasawara, Amirali Abdullah, Cullen Anderson, Narmeen Fatimah Oozeer, Jeff M. Phillips
Abstract
Binary-choice truth benchmarks ask models to choose between a correct and an incorrect answer, but if the two answers differ systematically in surface-level features, models can exceed chance without performing the intended reasoning. We show that this failure mode is detectable and can be exploited by downstream classifiers. In TruthfulQA, a simple six-feature logistic classifier achieves substantial accuracy in separating correct from incorrect answers. We further show that similar surface-level artifacts are present in additional benchmarks. To counteract this, we developed a general mechanism to clean them by removing the most leakage-reinforcing pairs. We release a version of TruthfulQA with surface-feature leakage reduced close to chance and provide a mechanism, Audit-Prune, so that the datasets can be cleaned before release.