Worker-pool orchestration improves cloud text classification with large language models
Agentic Autoscaling through Worker-Pool Orchestration for LLM-driven Text Classification in Cloud Computing Environments
Distributed, Parallel, and Cluster Computing
Summary
Processing large amounts of text using big language models can be slow and costly when lots of requests come in at once. The authors created a system that automatically adjusts computing resources by managing a pool of worker agents to handle these requests efficiently. This system works with different language models without extra changes and balances speed and accuracy well. Tests show it uses resources better and keeps good accuracy compared to fixed setups. This helps run large language model text classification smoothly on the cloud without wasting money or time.
What this means in practice
- •For cloud infrastructure teams: Automatically manage computing resources for scalable large language model text classification workloads in cloud environments.
- •For natural language processing engineers: Integrate elastic worker pools to efficiently handle bursts of text classification tasks using various LLMs without changing autoscaling logic.
Authors
Bablu Kumar, Anshul Verma, Rajkumar Buyya
Abstract
The growing adoption of large language model (LLM)-based systems for large-scale text processing has created a critical need for dynamic autoscaling to manage high-latency, bursty, and computationally intensive workloads. This paper proposes an agentic autoscaling framework through worker-pool orchestration for LLM-driven text classification. The framework integrates a priority task queue, a dynamic pool of agent workers, a real-time metrics collector, and an application-layer autoscaler. Its classifier-agnostic design supports both zero-shot and fine-tuned language models without modifying the autoscaling logic. The framework is evaluated using Autoscaling+BART and Autoscaling+DeBERTa against static allocation and standalone RoBERTa and DistilBERT baselines. On the AG News dataset, Autoscaling+BART achieves 84.5% accuracy, while Autoscaling+DeBERTa improves it to 90.5%. On the SMS Spam Collection dataset, Autoscaling+DeBERTa achieves 99.5% accuracy, whereas Autoscaling+BART attains 84.5% accuracy with lower execution time. Overall, the proposed framework consistently outperforms the baseline approaches in resource efficiency while maintaining high classification performance, demonstrating that elastic worker-pool orchestration provides an effective and cost-efficient solution for scalable LLM-driven text classification in cloud environments.