Syntax improves topic diversity in action focused text analysis
Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences
Computation and Language
Summary
People use topic models to find hidden themes in lots of written text. The authors created a new model called SCPTM that looks at sentence structure (syntax) to help find these topics. They tested it on different kinds of texts and found that using syntax helps make topics more varied and better captures how people express opinions or actions, but it doesn't always help in technical or formal documents. Most improvements came from a new way of encoding the text, while syntax was more useful in debates or argumentative writing. So, syntax matters mostly when the text is about actions and arguments, not just facts or administrative information.
topic modelingsyntactic dependencygraph attention networkvariational autoencoderlatent themesbag-of-wordsdocument embeddingstopic coherencepredicate-argument structurecomputational social science
Authors
Alessandro Meneghini
Abstract
Topic modeling is widely used in computational social sciences to identify latent themes in large text corpora. Traditional approaches rely on Bag-of-Words representations and generative models such as LDA, while recent methods like BERTopic operate on dense document embeddings. This paper introduces the Structural Contextual Probabilistic Topic Model (SCPTM), an architecture that incorporates syntactic dependency relations into topic inference. SCPTM represents a corpus as a heterogeneous graph of documents and words connected by lexical and syntactic edges, processed through a Graph Attention Network within a Variational Autoencoder to produce probabilistic, mixed-membership topic distributions. We evaluate seven topic modeling techniques (including four SCPTM ablations) across four corpora differing in register and discourse structure. Our framework combines coherence (C_V, C_NPMI), topic diversity, clustering-label alignment (NMI), and phrase-level diagnostics (complementarity and valence gap). Results show that SCPTM's neural architecture yields substantial gains in document-topic alignment over generative baselines, but these gains are attributable to the variational encoder rather than to syntax. Syntax contributes to topic diversity, where graph-augmented variants outperform the no-graph baseline across all corpora, and to descriptor quality: dependency paths capture predicate-argument structures and stance in deliberative registers, while proving redundant in technical and institutional corpora. The valence gap is positive across all variants, but driven primarily by phrase grouping rather than syntactic filtering. We conclude that syntactic encoding matters conditionally: it benefits action-oriented, argumentative texts, but introduces noise in informational or administrative registers.