Serverless autoscaling strategies compared for bursty workloads
A Unified Queueing-Based Framework for Comparative Evaluation of Serverless Autoscaling Strategies under Bursty Workloads
Distributed, Parallel, and Cluster Computing
Summary
Handling sudden bursts of user requests is hard for serverless computing platforms. The authors compare three different ways these platforms can automatically add or remove computing resources to keep services running smoothly. They created a model to test these methods fairly without using any specific commercial platform. Their findings show a trade-off: one method is fastest under normal loads but struggles when demand peaks, another is sensitive to settings, and a third keeps service quality more stable across all conditions.
What this means in practice
- •For cloud platform engineers: Choose and configure autoscaling methods to balance delay and stability under fluctuating workloads using this modeling framework.
- •For data center operators: Use insights from autoscaling comparisons to optimize power consumption without sacrificing performance for serverless applications.
Authors
Zahra Zafarzade, Ehsan Ataie
Abstract
Serverless computing platforms must preserve quality of service (QoS) while adapting to highly bursty and unpredictable workloads. To cope with workload fluctuations while satisfying QoS requirements and service-level agreements (SLAs), these platforms rely on automatic resource scaling. Commercial function-as-a-service (FaaS) platforms employ different autoscaling strategies to dynamically adjust the number of active function instances and the available processing capacity. In this work, three representative autoscaling strategies are modeled and compared: scale-per-request (SPR), concurrencylevel scaling (CLS), and metric-based scaling (MBS). To enable systematic evaluation independent of proprietary platform-specific implementations, we develop a unified modeling framework that combines analytical queueing network (QN) modeling with queueing Petri nets (QPN) implemented in Java Modeling Tools (JMT) to evaluate autoscaling behavior under two-phase hyper-exponential (H2) arrivals. Evaluation is conducted using four system-level metrics: throughput, mean response time, system population, and system power. The framework captures dynamic function instance (FI) activation, concurrency constraints, and queue buildup under saturation. Comparative analysis reveals a clear trade-off between peak performance and stability: SPR provides the lowest delay under moderate load but becomes fragile near capacity limits; CLS is governed by concurrency tokens and is highly sensitive to configuration under bursty conditions; whereas MBS provides the most stable QoS across the workload range by keeping utilization below a critical region and preventing explosive queue growth under critical load.