Papers for

hpc system operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

CQTwin toolkit simulates HPC schedules for better workload decisions

CQTwin: A Digital-Twin Toolkit for Operational Analytics of HPC Schedulers

Abstract: HPC facilities collect years of job scheduler records, yet much of this operational data remains underused because accessible tools for turning it into actionable insight are scarce. We present CQTwin, an open-source digital-twin toolkit for operational analytics of HPC scheduling data. CQTwin addresses this gap through two complementary modes: offline trace replay and online event-driven simulation. Built on the Python-based CQSim discrete-event simulator, CQTwin combines a fidelity-validated, high-performance C++ engine with an interactive browser-based dashboard and a Snapshot Lab, enabling facility staff, operators, and researchers to characterize workloads, compare scheduling policies, and explore what-if scenarios. As an offline twin, CQTwin replays archived workload traces under alternative policies to show how the same workload would have unfolded under different scheduling decisions. As an online twin, it consumes job events as they arrive and projects how alternative policies would process the currently queued jobs, providing guidance for real-time decision-making. The dashboard supports the complete analysis workflow, including workload characterization, experiment configuration and monitoring, and multidimensional comparison of simulation results, allowing users to determine visually which policies best suit a workload and what trade-offs they entail. We validate CQTwin using production traces from the Argonne Leadership Computing Facility spanning 2016-2023.

Mon 19 OctDistributed, Parallel, and Cluster ComputingPerformance
The gist
Large computing centers have many records of how computing jobs were scheduled but lack easy tools to learn from this data. The authors present CQTwin, a software tool that can replay past workloads and simulate how different scheduling policies might change outcomes. It works both offline with stored data and online with live job events, helping operators compare policies and plan better decisions. They tested CQTwin with real-world data from a major research facility over several years.
Open → 2610.26609v1

Deep reinforcement learning shows mixed results for HPC job scheduling

Deep Reinforcement Learning for HPC Job Scheduling: A Statistical Evaluation

Abstract: Deep Reinforcement Learning (DRL) offers learned job selection for High-Performance Computing (HPC), but performance against heuristics alone does not establish value beyond action masking. Six DRL treatments are compared: Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Deep Q-Network (DQN), each with and without masking. Three heuristics and a seed-paired masked uniform random control provide baselines across ten seeds on two real Slurm (Simple Linux Utility for Resource Management) workload traces, one homogeneous and one heterogeneous, under a common non-parametric framework. All masked treatments complete evaluation, whereas unmasked treatments are truncated. On the homogeneous trace, no masked DRL treatment significantly improves average waiting time or average and maximum bounded slowdown over the control. On the heterogeneous trace, MaskablePPO improves all three metrics relative to the control. Its average waiting time is equivalent to that of the three heuristics within a 10% margin, and its maximum bounded slowdown is 22% lower than that of Last-Come, First-Served (LCFS). MaskablePPO has the leading average waiting-time rank among complete treatments on both traces, but Nemenyi comparisons detect no significant difference from MaskableA2C. Among masked variants, MaskablePPO has the lowest recorded mean training time, while MaskableDQN has the lowest mean decision latency. These findings distinguish completion from baseline-specific scheduling gains and support including a masked random control when assessing the value of learned selection.

Wed 14 OctArtificial Intelligence
The gist
Scheduling jobs in high-performance computing (HPC) systems is important to manage resources efficiently. The authors studied whether deep reinforcement learning (DRL) can improve this scheduling compared to standard heuristic methods. They found that for simpler, uniform job types, DRL does not significantly outperform basic approaches. However, when jobs vary a lot, a DRL method called MaskablePPO showed some improvement in waiting times and fairness, though it was close to heuristic methods. The study also highlights that including certain baselines is important to fairly evaluate these learning methods.
Open → 2610.21008v1

Mathematical model improves scheduling for large scientific workflows

Mathematical Modeling of a Cognitive Continuum Digital Shadow for Large-Scale, Cross-Facility Workflows

Abstract: We present the mathematical foundations of a \emph{Cognitive Continuum Digital Shadow} (CCDS), a decision-support layer between users and the cross-facility infrastructure---instruments, networks, data stores and compute centers---of exascale and post-exascale scientific workflows. The CCDS couples a state-space representation of the continuum with multistage stochastic programming, so that deployment scenarios can be explored and optimized \emph{before} jobs are launched. This allows operators and users to quantify the cost, makespan and energy trade-offs of a workflow under uncertain resource availability, and hedge their decisions accordingly. We formulate the underlying optimization as a multimode, resource-constrained, stochastic supply-chain network design problem and demonstrate it on a realistic genomics workflow scheduled across heterogeneous HPC and data-center resources. This is the first of three papers; the second treats the underlying software architecture and the third reports large-scale use-cases.

Mon 7 SeptPerformance
The gist
Large scientific workflows often run across many computing centers and instruments, making it hard to plan and manage resources efficiently. The authors present a mathematical approach called the Cognitive Continuum Digital Shadow (CCDS) that helps users and operators understand and predict trade-offs like cost, time, and energy before starting jobs. This approach uses advanced optimization to handle uncertain availability of resources, allowing better decisions on where and when to run parts of a workflow. They demonstrated this method on a genome analysis example using diverse computing resources.
Open → 2609.07275v1