Generative and discriminative models vary in assigning topics to ecommerce content
Multi-Label Topic Assignment via LLM Distillation: A Comparative Analysis of Generative vs. Discriminative Student Models
Machine LearningComputation and Language
Summary
Assigning multiple topics to user-generated content like product reviews or conversations is tough because the language is informal and topics change quickly. The authors studied different small AI models to see which work best when copying labels from large language models. They found simple generative models do better on chat-like conversations, while discriminative models excel on structured reviews. Generative models also handle lots of topics and rare labels more reliably. Their findings help build faster, accurate topic classification systems for large online marketplaces.
What this means in practice
- •For ecommerce platform engineers: Deploy optimized topic classifiers to tag product reviews and buyer-seller chats with many evolving categories at low latency in large marketplaces.
- •For customer service automation teams: Implement generative student models to accurately assign detailed topics in multi-turn conversational data for better chat analysis and routing.
Authors
Sourabh Kasliwal, Shubhranshu Singh
Abstract
Multi-label topic assignment for user-generated content (UGC) -- including product reviews and buyer-seller conversations -- poses unique scalability challenges in large-scale e-commerce due to informal language, extreme label sparsity, and rapidly evolving taxonomies. While utilizing Large Language Models (LLMs) as labeling oracles to distill ground-truth data has emerged as an industry standard to bypass prohibitive manual annotation costs, determining the optimal, low-latency architecture for the resulting student models remains an open challenge. To address this, we conduct a comprehensive evaluation across Small Language Model (SLM) parameter scales (1B, 4B, and 8B) and architectural paradigms (causal generative versus bidirectional discriminative). Comparing generative text-to-label classifiers against discriminative baselines (DeBERTa-V3 and ModernBERT), our analysis reveals a crucial data-dependent trade-off: while discriminative models outperform ultra-lightweight generative models on structured product reviews, even the smallest 1B generative model surpasses discriminative baselines on complex, multi-turn conversational data. Furthermore, generative models maintain robust performance under massive label-set expansion (up to 112 topics) and severe long-tail distributions, whereas discriminative baselines suffer a 35% drop in Macro-F1 at scale. Finally, we detail the successful production deployment of these optimized models across both product review and conversational domains, demonstrating strict latency compliance and tangible business impact at a global marketplace scale.