On Topology's Role in ML Training Performance
2026-08-03 • Networking and Internet Architecture
Networking and Internet Architecture
AI summaryⓘ
The authors studied how two common network designs, fat-tree Clos and torus, affect the speed of important communication tasks in machine learning training systems. They analyzed which network performs better for these tasks, considering factors like network failures and how jobs are assigned to machines. Their findings show that no single network is best in every situation, but the fat-tree Clos usually finishes tasks faster and offers better flexibility and reliability. This work helps understand the trade-offs when choosing network layouts for machine learning hardware.
fat-tree Clostorus networkcollective communicationmachine learning workloadscompute acceleratorsnetwork topologyjob placementnetwork resiliencecommunication performance
Authors
Sarah McClure, Tegan Wilson, Brad Karp, Michael Mitzenmacher, Sylvia Ratnasamy, Scott Shenker, Minlan Yu
Abstract
Modern machine learning training workloads run on large-scale networks of compute accelerators. The networks commonly deployed in these systems are typically variations of two basic topologies: the fat-tree Clos and the torus. In this paper, we derive analytical results the elucidate how the choice of topology shapes achievable performance for the small set of collective communication operations that underlies modern machine learning workloads. We also consider how these results change when we include additional factors such as network failures and job placement strategies. Overall, we find that one topology does not dominate in all cases, but that the Clos achieves better collective completion time in most cases and provides benefits in resilience and flexibility.