LLM agents help analyze languages for patterns and rules faster
LLM Agents as Computational Typologists
Computation and Language
Summary
Studying many languages to find common patterns is hard and takes a lot of expert time. The authors created AUTOTYPOLOGIST, a computer program that uses a type of artificial intelligence called a large language model (LLM) to read language descriptions and examples. This program can pick out important parts, understand language data, and test ideas about language features automatically. It works well when given detailed descriptions but finds it harder when only short example texts are available. The tool shows promise for helping language experts do big comparisons, but experts still need to check its work.
linguistic typologylarge language model (LLM)reference grammarinterlinear glossed text (IGT)typological featurestypological universalsReAct workflowcrosslinguistic comparisonartificial intelligenceevidence-grounded analysis
Authors
Changbing Yang, Christopher Hammerly, Freda Shi, Jian Zhu
Abstract
Linguistic typology relies on expert analysis of reference grammars across languages, making large-scale crosslinguistic comparison labor-intensive and unscalable. We introduce AUTOTYPOLOGIST, an LLM agent for evidence-grounded typological analysis over reference grammars. The agent is capable of retrieving relevant grammar sections, analyzing interlinear glossed text (IGT), and iteratively reasoning over typological hypotheses using a ReAct-style workflow. We evaluate the system on TYPOLOGICAL FEATURE CODING against expert annotations and TYPOLOGICAL HYPOTHESIS TESTING with typological universals using 25 open-source reference grammars. Operating under different information constraints in TYPOLOGICAL FEATURE CODING, the agent can synthesize information from reference grammar prose but still faces challenges with only IGTs in the target language. In TYPOLOGICAL HYPOTHESIS TESTING, the agent can synthesize crosslinguistic evidence and identify both supporting cases and counterexamples. These findings suggest that LLM agents can support scalable and inspectable typological analysis, while still requiring expert validation.