Priority aware scheduler improves serverless function startup and balance
Priority-Aware Scheduler for Serverless Functions Provision and Load Balancing
Distributed, Parallel, and Cluster ComputingSoftware Engineering
Summary
Serverless computing lets programs run in the cloud without managing servers, but starting these programs slowly (cold starts) hurts performance. The authors noticed that serverless functions are called in patterns that can predict which ones to keep ready and which to remove. They created a scheduling system called SPES-Ext that uses these patterns and prioritizes functions to reduce cold starts, save memory, and balance workload better across workers. Their tests show big improvements in startup speed, memory use, and load distribution compared to previous methods.
What this means in practice
- •For cloud platform operators: Improve serverless system performance by reducing cold start delays and balancing load across memory-limited workers.
- •For data center infrastructure teams: Optimize resource usage and workload distribution in cloud environments by using priority-aware scheduling based on function invocation patterns.
Authors
Cheryl Lee, Zishan Su, Zhouruixing Zhu, Tianyi Yang, Yuxin Su, Pinjia He, Michael R. Lyu
Abstract
Serverless computing has emerged as a pivotal cloud deployment paradigm due to its efficient utilization of on-demand resources. However, the persistent cold start problem continues to hinder system performance. Current strategies often employ oversimplified preloading/unloading mechanisms that inadequately exploit invocation patterns, failing to optimize the critical trade-off between startup latency and resource efficiency. Furthermore, existing approaches disregard practical constraints of memory-limited multi-worker environments, exacerbating function eviction and load imbalance issues. To address these limitations, we present SPES-Ext, a rule-based priority-aware scheduler designed for memory-constrained distributed worker nodes. Our methodology stems from two key observations: (1) serverless systems exhibit concentrated invocation patterns enabling behavioral predictability, and (2) these patterns permit function categorization for granular instance management. This insight posits that characteristic-driven prioritization enables optimal scheduling rules, leading to our system design, which integrates three coordinated components: predictive function provisioning, eviction optimization, and dynamic load balancing. Experimental validation demonstrates the superiority of SPES-Ext over state-of-the-art solutions: 80% reduction in $75^{th}$-percentile cold starts, 83.92% decrease in memory waste, and 52.49% improvement in worker load variance.