Topic modeling improved with interpretable semantic features over words
Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features
Computation and LanguageHuman-Computer Interaction
Summary
Topic models try to summarize lots of text by finding the main themes or topics in it. Usually, these themes are described by showing the most important words, but this can miss some of the bigger ideas. The authors created MonoTM, which uses special features that are easier to understand than just single words. MonoTM separates finding what topics are in a document from how those topics are described, making the topics more meaningful and useful for exploring text collections.
What this means in practice
- •For text analytics teams: Produce clearer topic descriptions using meaningful semantic features instead of just top words for better text corpus analysis.
- •For information retrieval engineers: Enhance topic inference in document collections by leveraging full sparse autoencoder feature representations separately from topic interpretation.
Authors
Una Joh, Bei Yu
Abstract
Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce \textbf{MonoTM}, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document--topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.