Text classification adapts dynamically to reduce forgetting and speed training
Task-Oriented Rank Adaptation for Continual Learning in Text Classification
Computation and Language
Summary
Text classification systems often forget what they learned before when trained on new tasks, and sometimes knowledge from one task can harm another. The authors propose a method called TORA that cleverly decides when to reuse knowledge from similar past tasks or isolate new tasks to avoid confusion. This approach helps the system learn many tasks in a row without losing accuracy or wasting time. It works automatically, without needing to know the order of tasks beforehand.
What this means in practice
- •For natural language processing teams: Improve multi-task text classification models by dynamically sharing knowledge only among compatible tasks to maintain accuracy and reduce training time.
- •For automation platform developers: Enable efficient integration of sequential text analysis tasks without needing to predefine task order or identities, supporting flexible deployment.
Authors
Rey Sanchez Lopez, Eduardo Morales Manzanares, Hugo Jair Escalante
Abstract
Continual learning (CL) in text classification faces two critical challenges: catastrophic forgetting and negative transfer across sequential tasks. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA enable efficient adaptation by learning low-rank updates of the model parameters. However, these compact representations are normally trained in isolation, limiting their reuse across related tasks. We introduce Task-Oriented Rank Adaptation (TORA), a geometric routing framework that leverages the low-rank structure of LoRA adapters to decide whether to transfer knowledge from the most compatible expert (Boosting) or isolate the new task (Shielding) based on structural similarity. Evaluated across 15 diverse text classification benchmarks, TORA consistently avoids harmful routing decisions: compatible tasks exceed their isolated performance while reducing training time, and structurally distant tasks are protected from interference with no loss in accuracy. With a single geometric threshold and no reliance on task identities or predefined sequences, TORA provides a simple and effective approach for dynamic adapter routing in sequential text classification systems.