Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes
2026-08-03 • Computation and Language
Computation and Language
AI summaryⓘ
The authors studied how to sort user questions into categories like math, coding, or general text using Large Language Models. They compared methods that need no extra training with ones that do, such as simple neural networks. Both methods worked well for easy tasks, but trained methods were better at trickier tasks, like telling apart Java from Python code. Interestingly, the training-free methods handled mixed or tricky questions more reliably. This shows different approaches have strengths depending on the task complexity and prompt type.
Intent classificationLarge Language ModelsTraining-free methodsTraining-based classifiersMLP classifiersLinear probesInternal representationsAdversarial promptsMixed-intent detection
Authors
Nan Chen, Zhouhao Yang, Soufiane Hayou
Abstract
Intent classification in Large Language Models (LLMs) involves categorizing user prompts into predefined classes. For instance, given a user prompt, the system must determine whether it primarily concerns mathematics, coding, or general text processing. Such classification enables routing prompts to specialized models optimized for specific domains, improving both accuracy and computational efficiency. In this work, we conduct a systematic study comparing training-free vs training-based approaches for intent classification. For this purpose, we consider two lightweight, training-free methods based on statistics of internal representations and compare them against MLP classifiers and linear probes. Our comprehensive empirical evaluation reveals that 1) Both training-free and training-based methods saturate easy benchmarks (mathematics vs. coding vs. natural language), 2) Training-based classifiers have an advantage on harder classification tasks (e.g. Java vs Python), and 3) Training-free methods are generally more robust to mixed-intent and adversarial prompts.