Synthetic linguistic data reveal insights into language use and models

Synthetic linguistic data

Computation and Language

Summary

Linguistic research often uses language data, but synthetic data generated by AI models are now becoming common. The authors explain how this synthetic data can be like an artificial plant nursery, divided into two kinds: collections that mimic natural language and controlled experiments that test specific language ideas. They show that studying synthetic data alongside real language data helps find similarities and differences, but understanding their meaning requires knowing how the data were produced. This approach adds a new way to study language using AI-generated examples.

What this means in practice

  • For nlp engineers: Use controlled synthetic data to test and refine language model behavior on specific linguistic features.
  • For language technology developers: Analyze patterns in large collections of synthetic text to better understand model outputs and improve language tools.

A position paper. It proposes an approach and reports no results.

Authors

Giuseppe Samo

Abstract

Synthetic linguistic data have become increasingly prominent in linguistic research, from the analysis of large collections of AI-generated texts to controlled experiments investigating grammatical and lexical preferences. This glossary entry extends Bresnan's (2016) distinction between the linguistic garden and the bush by introducing an artificial plant nursery as a metaphor for synthetic data. We distinguish between synthetic bush data, consisting primarily of corpus-like collections of model-generated language, and synthetic garden data, in which model outputs are elicited under controlled conditions to investigate specific linguistic hypotheses. We argue that synthetic data constitute a distinct object of linguistic investigation: comparing them with natural language data can reveal both shared patterns and divergences, but their interpretation requires careful consideration of the conditions under which they are generated.