Safe walltime policies improve supercomputer scheduling efficiency
Evaluating Runtime Prediction and Conservative Walltime Policies in HPC Systems
Distributed, Parallel, and Cluster Computing
Summary
Supercomputers run big tasks that need time limits called walltimes. People often guess these times too long to avoid stopping their jobs early, but that wastes resources. The authors created a framework to safely suggest shorter, more accurate walltimes based on predicted runtimes. Tested on millions of jobs from two supercomputers, their approach cut wasted time by about a quarter while keeping job failures below 1%. This helps supercomputers work better and faster.
What this means in practice
- •For supercomputer operators: Reduce wasted compute resources by deploying conservative walltime recommendations that minimize job overruns and improve scheduling efficiency.
- •For cloud compute schedulers: Improve task time-limit assignments in large cloud clusters by integrating safe walltime policies with runtime predictors to reduce early task termination.
Authors
Hector Romero Ugalde, Youssef Faqir-Rhazoui, Marlon Funk, Jesus Gorroñogoitia Cruz
Abstract
Efficient use of large-scale supercomputing systems depends on accurate runtime estimation and walltime selection. To avoid premature termination, users often overestimate walltime, leading to poor resource utilization and reduced scheduling efficiency. This paper presents a predictor-agnostic framework for safe walltime recommendation in production HPC environments. Rather than proposing a new runtime prediction model, the primary contribution is the SAFE decision layer, which converts predicted runtimes into conservative bounded walltime recommendations suitable for deployment. Runtime prediction is therefore treated as a modular component within the proposed framework. We evaluate the approach using large-scale production job histories from the Fugaku and NREL Eagle supercomputers, comprising approximately 24M and 11M jobs, respectively. Runtime prediction performance is first evaluated using representative predictors, including linear regression, neural networks, and a neural ensemble. The proposed SAFE framework is then evaluated using the NN ensemble predictor. Experimental results indicate that similarity-aware prediction substantially improves runtime estimation for workloads similar to previous executions. When combined with the proposed conservative walltime recommendation policies, these improvements enable walltime gap reductions of approximately 20 - 26% while maintaining job termination rates below 1% across both systems.