Representation of syntax in LLMs through the lens of linear distance and similarity-aware entropy
Computation and Language
Summary
The authors studied how well neural language models can recognize different types of grammar connections between words using a method called structural probes. Instead of just looking at overall accuracy, they checked how accurately each type of grammar link is found separately. They found that two things mostly explain the differences in accuracy: how far apart the connected words usually are, and how varied the roles of the connecting words are. These findings help us understand how language models internally represent grammar and how that relates to the way word meanings are organized in the model.
structural probesneural language modelssyntactic treesundirected attachment scoresyntactic relationslinear distanceembedding spaceentropylatent representations
Authors
Juan Pablo Vigneaux, Mary Kennedy, Khalil Iskarous, Robert Frank, Matilde Marcolli
Abstract
Structural probes were introduced by Hewitt and Manning to reconstruct syntactic trees from a neural language model's latent representations. They are evaluated by calculating the proportion of syntactic tree edges correctly reconstructed over an annotated corpus (as measured by undirected unlabeled attachment score). Here, we disaggregate this measure, considering undirected attachment score by label (UASL), which assesses the reconstruction accuracy of each syntactic relation separately, establishing important differences among relations that overlap linguistic distinctions. Moreover, we identify two factors that predict most of UASL's variability across relations: (i) the mean and dispersion of the linear distance (on a log scale) between the related words, and (ii) the diversity (similarity-aware entropy) of the syntactic relation's head. These results, which hold across a range of model sizes and architectures, shed light on the degree of abstraction of the representation of syntax in language models and the dependence of such representation on geometric properties of the embedding space.