Literary Non-Style in LLM-Generated Text
Computation and Language
Summary
The authors studied how texts made by large language models (LLMs) differ from human writing. They found that LLM-generated text has a unique style and uses word patterns differently than humans do. By looking closely at groups of words called n-grams, the authors saw that both style and meaning are affected in LLM text. They argue that you can't fully separate how something is written (style) from what it means (semantics) in these texts.
Authors
Cory Massaro
Abstract
Prior work on LLM-generated text has demonstrated quantitative and qualitative departures from text produced by humans. LLM-generated texts differ from human writing in style, resulting in a characteristic textual "feel," while the semantic range of LLMs is much restricted compared to that of humans. In this contribution, I note simple but consistent patterns in the statistical distribution of n-grams within LLM-generated text. Via qualitative analysis of these n-grams, I reveal deficiencies in LLM style. Because higher-order n-grams correlate to semantic content, I conclude that questions of style and semantics are not cleanly separable.