Summary
High consistency in how large language models (LLMs) label text does not mean those labels truly reflect the real concerns people express. The authors studied comments and surveys about AI safety from European stakeholders and found that while LLMs agree very closely with themselves, their results don’t always match what people say in surveys. Different groups, like businesses versus public authorities, show varying differences between their text and survey answers. Also, people in nearby countries tend to have similar text-based opinions about AI risks. This shows that being consistent is not the same as measuring the right thing.
What this means in practice
- •For policy analysts: Assess differences between survey responses and LLM-analyzed text to refine AI policy feedback measurement.
- •For natural language processing teams: Improve validation methods to separate consistent LLM annotations from true measurement of communication constructs.
Tested on one dataset.
Abstract
High annotation reproducibility does not necessarily imply that an LLM-inferred measure captures the construct it is intended to measure. We test this distinction using a dataset from the European Commission's AI Act consultation, linking structured survey responses to free-text consultation submissions from the same stakeholders. LLM annotations of consultation submissions are highly reproducible (intraclass correlations > 0.99), yet show limited convergence with survey-reported measures of the nominal construct they were intended to approximate. Divergence between survey-and LLM-inferred text-based measures varies systematically across stakeholder groups: business associations express greater concern about AI risks in text-based consultations than in survey responses ({g} = +1.0), whereas public authorities and several nonbusiness groups show smaller or negative divergences. Divergences between scores suggest positive spatial autocorrelation across European countries (Moran's I = 0.347, p = 0.036), indicating that stakeholders from neighboring countries tend toward more similar text-based stances towards AI safety concerns. Despite divergence, survey-reported concerns remain strongly associated with support for explainability across all divergence levels. These results demonstrate that LLM annotation reproducibility can coexist with poor construct correspondence and motivate validation procedures that distinguish reproducibility, construct validity, and communication context variation when LLMs are used as measurement instruments.