Large language models sometimes give another state’s policy value

Right Number, Wrong State? Measuring Cross-Jurisdiction Substitution in LLM Recall of State Policy

Computation and Language

Summary

When a large language model (LLM) answers questions about state-specific policies, it can sometimes give a number that is correct but for the wrong U.S. state. The authors tested this across all states for specific Medicaid income eligibility numbers. They found that LLMs like Claude Sonnet 5.5 and GPT-5.6 Sol often repeat values from different states instead of making up numbers, but identifying these swaps is tricky because sometimes the same state’s data appears under different formats or years. So, careful comparison with a complete reference for each state is needed to understand these errors.

What this means in practice

  • For policy data analysts: Improve validation processes to detect when AI outputs use correct numbers from different states instead of hallucinating values.
  • For ai application developers: Design systems to cross-check state-specific outputs from LLMs against full reference sets to avoid misinterpreting regional policy data.

Authors

Jiayu Feng

Abstract

When an LLM answers a state-specific policy question wrongly, it may be hallucinating, or it may be returning a real value that holds in another state. We test this with a minimal-set design: the question wording is fixed and only the jurisdiction varies, across the 50 U.S. states and the District of Columbia (51 jurisdictions) and three exactly defined Medicaid income-eligibility quantities. Gold values come from an official data book and agree with an independent source in 101 of 102 checked cells. Under a pre-registered protocol, Claude Sonnet 5.5 and GPT-5.6 Sol reproducibly give another state's current value, identical across two independent repeats, for 10 and 25 of 153 items. Attribution is fragile, however. Crediting any wrong answer that equals another state's value yields 3-5x more reproducible substitutions than checking every number in the asked state's own records, because many apparent cross-state answers are the asked state's own values under another convention or from an earlier year. Claims about cross-jurisdiction error need a complete same-state reference set. We will release the protocol, gold table, and all model outputs.