Dual branch models improve multi label text classification accuracy
LLM-Enhanced Dual-Branch Learning for Large-Scale Multi-Label Text Classification
Computation and LanguageArtificial Intelligence
Summary
Classifying documents with thousands of possible labels is a tough challenge because each document relates to only a few labels. The authors introduce DualMLC, a system that uses two different types of language models to analyze the same text separately and then combines their predictions. This method helps capture different clues from the text that a single model might miss. Their approach outperforms previous systems on multiple large datasets for multi-label text classification. The source code is publicly shared to encourage use and further research.
What this means in practice
- •For content moderation teams: Automatically assign multiple content categories from a large label set to user-generated text for improved content filtering accuracy.
- •For enterprise document management teams: Improve tagging of large document collections with relevant labels from extensive vocabularies, enhancing search and retrieval.
Authors
Hui Ye, Jing Zhang, Xiulong Yang, Rajshekhar Sunderraman
Abstract
Large-scale multi-label text classification assigns a small subset of relevant labels to each document from a vocabulary containing thousands or tens of thousands of candidate labels. Although pretrained language models have improved semantic text representations, most representation-based approaches center their prediction pipelines on a primary encoder or combine auxiliary features within a single ranker. The complementarity between heterogeneous language models therefore remains insufficiently explored. We propose DualMLC, a dual-branch framework that processes the same document through an autoregressive decoder-only language model and a bidirectional encoder. Each branch maintains its own representation pathway and independently estimates relevance scores over the shared label space. DualMLC combines the two score vectors through late logit fusion, allowing shared evidence to reinforce relevant labels and branch-specific evidence to compensate for limitations in the other branch's representation. DualMLC achieves state-of-the-art results on three widely used large-scale multi-label text classification benchmarks. Ablation results further confirm that integrating the heterogeneous predictors produces stronger rankings than either branch alone. The source code is publicly available at https://github.com/huiyegit/DualMLC.