Deep reinforcement learning shows mixed results for HPC job scheduling
Deep Reinforcement Learning for HPC Job Scheduling: A Statistical Evaluation
Artificial Intelligence
Summary
Scheduling jobs in high-performance computing (HPC) systems is important to manage resources efficiently. The authors studied whether deep reinforcement learning (DRL) can improve this scheduling compared to standard heuristic methods. They found that for simpler, uniform job types, DRL does not significantly outperform basic approaches. However, when jobs vary a lot, a DRL method called MaskablePPO showed some improvement in waiting times and fairness, though it was close to heuristic methods. The study also highlights that including certain baselines is important to fairly evaluate these learning methods.
What this means in practice
- •For hpc system operators: Optimize job scheduling in HPC clusters with reinforcement learning techniques that perform competitively on workloads with varied job types.
- •For cloud infrastructure teams: Evaluate and integrate masked reinforcement learning algorithms to improve job queue management in heterogeneous computing environments.
Authors
Justin M. Cheney, Clement N. Nyirenda
Abstract
Deep Reinforcement Learning (DRL) offers learned job selection for High-Performance Computing (HPC), but performance against heuristics alone does not establish value beyond action masking. Six DRL treatments are compared: Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Deep Q-Network (DQN), each with and without masking. Three heuristics and a seed-paired masked uniform random control provide baselines across ten seeds on two real Slurm (Simple Linux Utility for Resource Management) workload traces, one homogeneous and one heterogeneous, under a common non-parametric framework. All masked treatments complete evaluation, whereas unmasked treatments are truncated. On the homogeneous trace, no masked DRL treatment significantly improves average waiting time or average and maximum bounded slowdown over the control. On the heterogeneous trace, MaskablePPO improves all three metrics relative to the control. Its average waiting time is equivalent to that of the three heuristics within a 10% margin, and its maximum bounded slowdown is 22% lower than that of Last-Come, First-Served (LCFS). MaskablePPO has the leading average waiting-time rank among complete treatments on both traces, but Nemenyi comparisons detect no significant difference from MaskableA2C. Among masked variants, MaskablePPO has the lowest recorded mean training time, while MaskableDQN has the lowest mean decision latency. These findings distinguish completion from baseline-specific scheduling gains and support including a masked random control when assessing the value of learned selection.