DeepShare improves GPU cluster use by coordinating tenant needs
DeepShare: Assurance-Driven Deep Learning Job Scheduling for Multi-Tenant Clusters
Distributed, Parallel, and Cluster Computing
Summary
Many companies share powerful GPU computers for running AI jobs, but these shared GPUs often sit idle even when jobs are waiting. The authors designed DeepShare, a smart system that watches how each tenant uses GPUs and adjusts their access in real time to keep the GPUs busy while making sure jobs don’t get delayed too long. By using this approach, DeepShare makes better use of GPUs, speeds up job completion, and balances fairness for all users sharing the cluster.
What this means in practice
- •For cloud infrastructure teams: Improve GPU cluster efficiency and reduce job wait times by dynamically managing tenant resource use with assurance-based scheduling.
- •For data center operators: Increase GPU utilization while meeting tenant service guarantees by coordinating quotas, preemption, and sharing using a continuous tenant-assurance signal.
Authors
Jinghao Wang, Yihang Zhou, Xiao Zhou, Xinlei Zheng, Xiaoyang Sun, Tianyu Wo, Chunming Hu, Renyu Yang
Abstract
Multi-tenant GPU clusters frequently remain underutilized even when tenants experience long queueing delays, because quota control, queue ordering, preemption, and GPU sharing are driven by different local signals. We present DeepShare, a scheduler that uses a continuous tenant-assurance signal to coordinate these decisions at runtime. DeepShare combines elastic quota borrowing, tenant-specific runtime prediction, cost-aware best-effort preemption, and interference-aware MPS colocation, while using the same assurance signal to decide when borrowed capacity should be reclaimed and when sharing should become more conservative. In trace-driven experiments on 23,859 Venus jobs and 3,200 internal jobs, DeepShare achieves an average GPU utilization of 70.58%, a 29.5% improvement over the strongest non-intrusive sharing baseline, while reducing average queueing delay by 46%. On a 16-GPU Kubernetes testbed, it reduces the average job completion time by 34% and maintains 93% QoS compliance for guaranteed tenants. These results show that treating tenant assurance as a runtime control loop achieves a more advantageous utilization-QoS trade-off than optimizing quotas, scheduling, and resource sharing independently.