Adaptive transfer improves detecting unreliable outputs in language models
Adaptive Transfer of Representational Signatures for LLM Reliability Detection
Computation and LanguageMachine Learning
Summary
Large language models sometimes give unreliable or wrong answers, which can be hard to spot. The authors show a new way to use knowledge from related areas to better detect these mistakes, even with very little labeled data from the new area. Their approach combines different detectors by learning how well each relates to the new task, improving prediction accuracy significantly. This helps make language models safer and more trustworthy in diverse topics.
What this means in practice
- •For ai quality assurance teams: Improve reliability checks for language model outputs in new domains with minimal labeled data by adaptively combining signals from related tasks.
- •For automated customer support developers: Detect hallucinated or erroneous responses from language models tailored to specific product or service categories using this adaptive transfer method.
Authors
Yongtong Li, Yixuan Florence Wu, Jiawei Zhang, Naichen Shi
Abstract
Large language models (LLMs) are reported to be susceptible to reliability issues including hallucinations and reasoning errors. Existing quality detectors can predict unreliable outputs based on internal representations with decent accuracies. However, training these detectors often requires substantial domain-specific labeled data, which limits their applicability to new domains. We propose to address this challenge by transferring representational signatures across related domains. More specifically, we build a statistical framework that first extracts low-dimensional signatures for each domain, then uses cross-fitting to learn composition weights of domain-specific predictors. The weights adapt to each source domain's predictive relevance to the target domain. Our asymptotic analysis of a least-squares analogue characterizes the limiting prediction risk and aggregation weights in terms of domain discrepancies and representation bias. Experiments across multiple hallucination datasets and open-weight LLMs show that, with only 25 labeled questions in the new domain, adaptive transfer improves hallucination-detection AUROC by 6.8%. For reasoning-step prediction, it improves AUROC by 8.4%.