Vietnam province reform inspires new Vietnamese language benchmark
ViAdminShift: A Vietnamese Benchmark for Event-Level Knowledge Updates Built from the 2025 Administrative Reform
Machine LearningComputation and Language
Summary
Vietnam drastically reorganized its provinces and districts in 2025, changing many place names and administrative boundaries. The authors created a Vietnamese language dataset called ViAdminShift to test if computer models can update their knowledge to reflect these big changes. They collected questions, related Wikipedia texts before and after the reform, and tables showing old and new place relations. Their tests show some models have learned the updates, but many still give outdated answers. The dataset and code are available for others to improve how AI remembers and updates facts about current events.
What this means in practice
- •For natural language processing engineers: Evaluate and improve Vietnamese language models’ ability to update factual knowledge about recent events like geographic reforms.
- •For government it teams: Maintain up-to-date administrative databases and services by integrating tools tested with ViAdminShift to detect and correct outdated geographic information.
Authors
Phan Binh Nguyen Lam, Nguyen Thai Anh
Abstract
On July 1, 2025, Vietnam reduced its 63 provinces to 34, abolished its 696 districts, and replaced its 10,035 commune-level units (wards) with 3,321. ViAdminShift, a Vietnamese language resource about this event, consists of 7,995 benchmark items, 2,281 retrieval passages from pre- and post-reform Wikipedia revisions and dated templates, and old-to-new tables that a released script rebuilds from the official conversion table (not redistributed). Changed items store the pre-reform answer (except 121 address items), a $2\times2$ design uses untouched units as controls, and 400 addresses appear without diacritics. The tables agree with an independent lineage on 10,564 of 10,565 official pairs, disagreements were settled against the resolutions, and the probing-track scorer agrees with both authors' blind labels (Cohen's $κ=0.91$ and $0.89$). Data, outputs, code, and a datasheet are released. In the probing track, at least 20 of 67 models know the reform. Fourteen of them give the new province alone for at most 11% of plain questions about old addresses (old province only: 14-100%); a bare date recovers up to 100%. Post hoc (80 province-free addresses, single runs), with reasoning off, today's date in the system prompt leaves 68-100% old-province-only answers for 10 of 12 models; with reasoning, an "as of" prefix in the question leaves none for seven of eight. On the 1,262-item benchmark subset, with same-source old and new passages, small models unaware of the reform give 14-30% outdated answers and the five benchmark-track knowers (also differing in size or provider) 0-5%.