Papers for

gpu cluster operators

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

RoboScope Kit speeds up robot simulation evaluations with unified interfaces

RoboScope Kit: Unified Interfaces for Fast and Scalable Embodied-AI Evaluation

Abstract: Simulation benchmarks are the standard way to compare robot policies, but each benchmark brings its own evaluator, observation layout, and action convention, and evaluation is slow: 500 RoboTwin Easy episodes with one simulator and one model on one GPU take 359.7 minutes. Profiling the official StarVLA evaluator shows that simulation occupies CPU cores while the GPU averages 14.3% utilization. RoboScope Kit wraps each model once in a Python class called a Policy, and each benchmark once in an adapter around the benchmark's own evaluator, so the benchmark's rules and scoring stay unchanged. Scheduling runs simulator workers and Policy replicas as separate pools, sizes both from measured probes, and shards work deterministically across GPU instances; runtime optimization renders only the observations an action-chunk Policy reads. Adapters cover six benchmark families. On LIBERO-Long with a StarVLA checkpoint on one 48 GiB RTX 4090, the concurrency search selects twelve simulator workers sharing three Policy replicas, which evaluate 500 episodes 4.45$\times$ faster than the official StarVLA evaluation with equivalent success rates, 2.78$\times$ faster than RLinf, and 1.52$\times$ faster than vla-eval at its tuned configuration. With CUDA MPS, which lets the replicas run their kernels concurrently, the search selects sixteen workers sharing four replicas, 6.16$\times$ faster than StarVLA, 3.85$\times$ faster than RLinf, and 2.10$\times$ faster than vla-eval. On RoboTwin Easy, four GPUs with chunk-boundary observation complete the 500 episodes in 14.4 minutes, 25.04$\times$ faster than the single-GPU baseline, with identical success outcomes and terminal steps.

Mon 2 NovRoboticsArtificial Intelligence
The gist
Testing how well robot software works usually takes a long time because each test uses different tools and setups. The authors created RoboScope Kit, which acts like a universal adapter to make these tests faster and more organized without changing how the tests are scored. By smartly managing how many simulators and software copies run together, their system runs robot tests up to 25 times faster while keeping the same results. This helps teams compare robot programs quickly and more efficiently on standard computers with GPUs.
Open → 2611.03850v1

Ineffective gpu use in clusters cut by scheduling based on readiness prediction

Ineffective Occupancy in Heterogeneous GPU Clusters: Prediction, Contribution Analysis, and Readiness-Aware Scheduling

Abstract: In heterogeneous GPU clusters, GPUs are costly and scarce, yet placing a job on a machine does not mean that the job becomes ready to run. A large amount of allocated capacity can remain occupied for long periods without producing useful computation, further aggravating the waste of compute resources. On the Alibaba GPU cluster trace we find that, even when GPU capacity is sufficient and placement is complete, 52.70% of execution records never became ready. We term this phenomenon ineffective occupancy. Building on this diagnosis, and keeping the placement policy fixed, we propose Readiness-Aware Scheduling (RAS): it uses only nine attributes observable at admission time to predict readiness probability and reorders admission accordingly. Under a five-fold split grouped by machine, the gradient-boosted model reaches an ROC-AUC of 0.8187. A feature-group contribution analysis shows that the model's score is driven mainly by job profile, duration, and GPU model, with the request-size group contributing least, followed by the scheduling and concurrency group. In GPU terms, RAS reduces the ineffective-occupancy ratio (the share of cluster GPUs held under ineffective occupancy) from 43.64% to 27.15% and raises the effective coverage of large requests (eight GPUs or more) from 46.11% to 63.18%, without sacrificing capacity utilization. Relative to the upper bound obtained by ordering with ground-truth labels, RAS captures 65.1% of the attainable improvement, well above the interpretable rules built only from the contribution-analysis conclusions; the gain therefore comes mainly from predicting with all nine attributes jointly rather than from combining rules built on any single attribute group. These results indicate that readiness status is a measurable, predictable, and optimizable dimension of admission decisions in heterogeneous GPU clusters.

Sun 18 OctDistributed, Parallel, and Cluster Computing
The gist
In GPU clusters used for running jobs, GPUs can be wasted when jobs are assigned but never actually ready to run. The authors call this problem ineffective occupancy and found it happens a lot in a big Alibaba cluster. They created a method called Readiness-Aware Scheduling (RAS) that predicts whether a job will be ready based on nine features before starting it. By using RAS, much less GPU time is wasted by jobs that never run, making the cluster work better without losing capacity. Their method works well by combining these features rather than using simple rules about one characteristic alone.
Open → 2610.25066v1

Dynamic metering system measures actual compute in heterogeneous GPU clusters

From Nominal Capacity to Deliverable Compute: Design, Calibration and Validation of the D-PIN SCU Dynamic Metering System

Abstract: Nominal chip specifications do not capture the compute deliverable by heterogeneous distributed networks under communication, runtime and verification constraints. We formalize D-PIN Standardized Compute Units (D-PIN SCU) as a dynamic metering system for deliverable compute. The system combines five-dimensional resource normalization with topology-aware and runtime-corrected metering. A cluster-scale Roofline model yields a closed-form communication ridge, while runtime factors describe thermal, fault and load effects. We show that runtime deviations cannot be identified from nominal specifications alone. Uniform re-anchoring preserves relative metering results, and a simplex-constrained calibration algorithm estimates resource weights. The framework couples Proof-of-Physical-Work verification to a no-arbitrage condition for over-claiming and electricity-aware pricing. An end-to-end recomputation on the Zhangjiang Science Gate production node covers ten heterogeneous 8-GPU servers totaling 59.5 D-PIN SCU. Billed throughput of approximately 33,000 D-PIN SCU-hours is consistent with the physical upper bound at 78% utilization over 720 hours. Derived metrics are 3.02 kW per D-PIN SCU at the supply-and-distribution design boundary and 0.26 D-PIN SCU-hours per delivered kWh. Empirically calibrated chip values differ materially from datasheet projections; the published calibration is not independently reproducible from vendor specifications. Validation is limited to one production node, and the originality claim concerns the integrated system rather than the SCU terminology or individual components. The framework provides a basis for scheduling, verification and pricing of deliverable compute in distributed heterogeneous networks.

Mon 12 OctDistributed, Parallel, and Cluster Computing
The gist
Chip specs usually tell you the max power, but don’t show how much compute you really get in complex systems where communication and other limits matter. The authors designed a method called D-PIN SCU that measures actual deliverable compute by combining resource use, network layout, and runtime conditions. They tested it on a large GPU cluster and found it matches real power and compute use well, showing differences from simple datasheet numbers. This system helps users schedule, verify, and price compute fairly in mixed hardware networks.
Open → 2610.16374v1

VarioPath speeds up GPU cluster data sharing by managing PCIe traffic

VarioPath: Workload-Aware All-to-All Communication for PCIe GPU Clusters

Abstract: AlltoAllv communication is a critical primitive in distributed large-model inference, particularly for mixture-of-experts (MoE) models. The growing adoption of PCIe GPU systems for cost-efficient inference makes AlltoAllv performance on these systems increasingly important. Without a dedicated scale-up interconnect (e.g., NVLink or Infinity Fabric), PCIe GPU systems carry both intra-node and inter-node traffic through the PCIe hierarchy, where concurrent transfers can contend for PCIe link bandwidth. This link contention, compounded by skewed traffic distributions and dynamic traffic demand, makes efficient AlltoAllv scheduling challenging. Existing approaches are either poorly suited to PCIe GPU systems or incur substantial schedule synthesis overhead that reduces their practicality in real-world deployments. We present VarioPath, an efficient AlltoAllv scheduling framework for PCIe GPU systems. It combines an offline topology-aware analyzer with an online demand-aware scheduler. The analyzer records contention-free transfer patterns as AlltoAllv channels and exploits topology symmetry to build a compact catalog for efficient search. The online scheduler decomposes each AlltoAllv invocation's demand across a sequence of channels, adapting to rapidly changing and skewed traffic while incurring low planning overhead. Evaluation on four platforms (up to 256 GPUs) shows average AlltoAllv speedups of 5.88x over FAST and 1.72x over DeepEP. End-to-end experiments show that VarioPath reduces Qwen3 inference latency by up to 27.2% and Wan2.1 generation latency by 6.1%.

Mon 28 SeptDistributed, Parallel, and Cluster ComputingNetworking and Internet Architecture
The gist
Sharing information quickly between many GPUs is important for running large AI language models smoothly. On systems using PCIe connections, the data transfer can get jammed because many transfers compete for the same routes. The authors designed VarioPath, a method that smartly plans these data transfers based on the computer setup and current demands. This approach helps reduce delays and speeds up tasks like AI model inference on GPU clusters.
Open → 2609.34340v1

Charm++ runtime enables fast no-restart resizing for cloud HPC

No-Restart Elasticity in an Adaptive Runtime System for Cloud-Native HPC

Abstract: Exploiting discounted spot instances for HPC requires an application to change its resource allocation at runtime, shrinking ahead of an interruption and expanding onto replacement capacity. Existing elasticity mechanisms implement rescaling as a full process teardown followed by a cold restart at the new processor count, and the restart stage accounts for up to 95% of the total overhead, growing with node count and increasing substantially on GPUs due to CUDA context initialization. In this paper, we present a no-restart rescaling mechanism for the Charm++ runtime system in which surviving processes never exit: a rescaling operation consists of a membership negotiation with a lightweight external coordinator, reconciliation of the UCX communication endpoints with the new cluster view, and a return to the top of the runtime initialization path that preserves live application state. This reduces the cost of a rescaling operation, excluding the load balancing step that any rescaling model requires, from multiple seconds to 8--15ms on CPUs and 7--11ms on GPUs, at 4 to 32 instances. Because processes survive, GPU device state persists in place, eliminating the checkpointing daemons previously required for GPU elasticity, and a launcher-independent bootstrap mechanism removes the dependence on supervised process managers, which are incompatible with spot instance interruptions. Integrated with an existing spot instance management framework, the mechanism cuts the end-to-end overhead of eight simultaneous interruptions to 0.2% of runtime on CPUs and 0.6% on GPUs, and raises the rate at which a job can be rescaled below 1% overhead by a factor of seven.

Sat 26 SeptDistributed, Parallel, and Cluster Computing
The gist
Running big computing jobs in the cloud often means using cheap but unreliable machines that can disappear suddenly. The authors present a way for these jobs to shrink or grow without the usual long pause to restart everything. Their method keeps all running processes alive and just updates connections and states quickly, saving seconds down to milliseconds. This helps programs using GPUs too, avoiding slow restarts related to GPU setup. Overall, their approach cuts downtime drastically, making cloud computing with spot machines smoother and cheaper.
Open → 2609.32217v1

Compass-abs reduces gpu cluster resource fragmentation for faster deep learning

COMPASS-ABS: Reducing Fragmentation in Shared GPU Clusters for Deep Learning Training Workloads

Abstract: With the rapid advancement of deep learning technology, shared GPU clusters receive an increasing number of deep learning training (DLT) jobs. Yet resource fragmentation make such clusters underutilized and forces the DLT jobs running on them to endure long turnaround times. Extensive research has been devoted to quantifying fragmentation and developing scheduling algorithms that alleviate its impact. However, existing fragmentation measures break down in the absence of workload distribution information, while current schedulers cannot continuously maintain resource fragmentation at a low level. To tackle these problems, we first introduce Scheduler-Induced Fragmentation (SIF), a metric built on the notion of partial-nodes that is independent of historical workload knowledge. We then propose COMPASS-ABS, which employs the COMPact-ASSured (COMPASS) algorithm to confine the cluster state within a tight Anchor-Based Space (ABS), whose construction fully leverages the topological alignment between dominant workload size and node capacity. Moreover. We also prove that it ensures SIF is bounded by $\frac{2}{N}$ under a workload composition condition that matches both theory and production. Evaluations implemented on a physical cluster and a simulated cluster demonstrate COMPASS-ABS effectiveness at improving resource utilization, reducing DLT job completion time by reducing fragmentation.

Wed 16 SeptDistributed, Parallel, and Cluster ComputingMachine Learning
The gist
Large clusters of GPUs are often used to train deep learning models, but their resources can become scattered and inefficiently used, causing delays. The authors introduce a new way to measure this problem, called Scheduler-Induced Fragmentation, that doesn’t rely on knowing past workloads. They then propose a method named COMPASS-ABS to keep the GPU resources organized by aligning job sizes with node capacities. Their approach helps make better use of GPUs and shortens the time it takes to complete training jobs.
Open → 2609.18519v1