TopicMerge combines fast and smart methods for better topic grouping
TopicMerge: A Divide and Conquer Approach to Topic Modeling
Computation and LanguageMachine Learning
Summary
Grouping documents into topics is hard because fast methods miss some details, while smart AI tools are too slow for large amounts of data. The authors came up with TopicMerge, which first breaks documents into many tiny, clear groups quickly. Then it uses a smart AI to carefully combine these small groups into bigger, meaningful topics. This way, TopicMerge is both accurate and fast, using way fewer AI calls than older methods.
What this means in practice
- •For content managers: Organize large collections of documents into clear topics efficiently without heavy computing resources.
- •For information retrieval teams: Improve search and categorization systems by building accurate topic groups from large text databases faster.
- •For legal tech developers: Build smarter document review tools that group legal bills or case files into meaningful topics with lower AI usage.$Commercial implications: Enables selling efficient legal document analysis software that works well and saves on costly AI computing.
Authors
Matthew Raffel, Lizhong Chen, Sai Munikoti, Sameera Horawalavithana
Abstract
Recent advancements in topic modeling have split into two main paths: efficient embedding-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model (LLM) approaches that achieve state-of-the-art clustering performance but at the cost of prohibitive computational expense and poor scalability. This paper introduces TopicMerge, a novel multi-stage pipeline that bridges these approaches to achieve both state-of-the-art clustering performance and computational efficiency. TopicMerge first employs an efficient clustering pipeline to deliberately over-segment a document corpus into a large set of fine-grained, semantically pure micro-clusters, which are then given initial semantic labels. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state-of-the-art clustering performance while reducing required LLM calls by over 92% compared to document-level methods, offering a practical and scalable solution.