Double deep q learning improves cloud autoscaling stability and efficiency
Stability-Aware Proactive Autoscaling Using a Double Deep Q-Network in Cloud Computing Environments
Distributed, Parallel, and Cluster Computing
Summary
Cloud computing needs to quickly add or remove resources to handle changing workloads without delays or wasted resources. The authors present a new method using a type of artificial intelligence called a double deep Q-Network, combined with a strategy to remove resources more carefully. This approach predicts future needs better and avoids constant adjustments up and down. Their tests show it reduces errors, uses the CPU more efficiently, and keeps the cloud system more stable over time.
What this means in practice
- •For cloud infrastructure teams: Improve resource allocation by deploying predictive autoscaling that reduces SLA violations and smooths scaling actions under dynamic workloads.
- •For devops engineers: Use advanced AI-driven scaling policies to minimize system disruptions and optimize CPU use during latency-sensitive cloud applications.
Authors
Bablu Kumar, Anshul Verma, Rajkumar Buyya
Abstract
Dynamic workloads and latency-sensitive applications require efficient autoscaling in cloud computing environments. However, most existing approaches rely on reactive mechanisms based on static thresholds, resulting in delayed responses and scaling oscillations under workload uncertainty. To address these limitations, we propose a double deep Q-Network-based proactive autoscaling approach (DDQN-Proactive) along with Resource Removal Strategy (RRS). The proposed (DDQN+RRS) enhances decision-making by decoupling action selection from value evaluation, enabling more stable and adaptive scaling. Experimental results demonstrate that the proposed method outperforms both reactive and existing proactive approaches. Specifically, DDQN+RRS achieves a lower Service Level Agreement (SLA) violation rate (11.81%), higher CPU utilization (52.23%), improved scaling stability, fewer scaling events (2,488), and reduced pod restarts (1,246). Furthermore, the approach ensures smoother autoscaling behavior by significantly reducing oscillations over time (0-60 s). While reactive methods exhibit substantial fluctuations in pod allocation, Reactive reduces these variations, and DDQN+RRS achieves the most stable and smooth scaling, particularly during the 15-30 s, 40-45 s, and 55-60 s intervals.