ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language
Machine Learning
Summary
The authors study how weird or false information in AI language model outputs can be detected by looking at differences in the hidden representations of the prompt (input) versus the response (output). They propose a new test called ASK-NN that compares these hidden states by focusing more on the prompt data, which is usually longer, using a nearest-neighbor graph approach. The test is mathematically sound, works efficiently, and performs well in experiments detecting artificial text and hallucinations in generated language. Their method can help understand and identify when AI models produce unreliable or artificial content.
Authors
Sergey Zakharov, Rodion Oblovatny, Alexey Zaytsev
Abstract
Hallucinations and artificial text in LLM-generated outputs often appear as distributional deviations between prompt and response hidden-state distributions. Since prompts or retrieved contexts typically serve as reference samples and responses as query samples, with major differences in length, these asymmetries motivate the use of change test statistics that treat the two samples differently. We consider an asymmetric two-sample test ASK-NN based on the directed k-nearest-neighbor graph. Our statistic counts reference points whose nearest neighbor in the pooled sample is also a reference point. Under the permutation null, it admits an exact finite-sample conditional mean and variance; we further establish asymptotic normality and consistency under fixed alternatives. ASK-NN is computationally effective and easy to implement. Empirically, it is competitive with kernel and graph-based baselines on synthetic benchmarks, artificial-text detection, and LLM hallucination detection from token-level hidden states.