Gpu cluster simulator improves training time predictions with network modeling
Accurate Simulation of Distributed Training Jobs with Network Contention Modeling
Distributed, Parallel, and Cluster ComputingNetworking and Internet Architecture
Summary
Predicting how long it takes to train AI models on multiple GPUs can be tricky because many simulators ignore slowdowns caused by network sharing. The authors created a simulator called MoSim that includes how jobs compete for network resources, leading to much more accurate predictions of training times. Their tool also avoids using actual GPUs to collect data, making the setup faster and easier. This work helps understand how scheduling affects training speed in GPU clusters.
What this means in practice
- •For cluster operators: Predict job completion times more accurately by simulating network contention effects in GPU clusters before scheduling workloads.
- •For cloud infrastructure engineers: Design and evaluate job scheduling strategies that consider dynamic network contention to optimize GPU cluster utilization.
Authors
Yeonho Yoo, Hyunho Lee, Hyunmok Choi, Chuck Yoo, Gyeongsik Yang
Abstract
Trace-driven simulation is widely used to evaluate distributed training (DT) jobs in GPU clusters, but existing simulators either ignore network contention or approximate it with a fixed penalty. This misses how scheduling decisions determine which jobs share server network interfaces and inter-server links, thereby changing networking time during training. As a result, our motivating experiments demonstrate that they incur large errors, reaching up to 73.64% mean absolute percentage error (MAPE) in average job completion time (JCT). This paper introduces MoSim, a GPU-cluster simulator that models DT job execution under dynamic network contention. MoSim combines GPU-free characterization with network contention model: it obtains each job's compute time, networking time, and networking volume without GPUs, then uses the current worker assignment to estimate how shared network interfaces affect each job's iteration time. Our evaluation shows that, compared with existing simulators, MoSim reduces simulation error for average JCT by up to 3.28$\times$, tail (99th-percentile) JCT by up to 7.79$\times$, and makespan by up to 8.48$\times$, while modeling NIC contention factors with only 8.63% error on average. By avoiding real-GPU profiling, MoSim also reduces input construction overhead by 44.6$\times$.