DragonFly plus network oversubscription ratio holds for broad traffic patterns
Understanding the oversubscription behaviour of DragonFly+ networks
Networking and Internet Architecture
Summary
Oversubscription happens when network demand exceeds capacity, causing slowdowns. The authors show that a known 2:1 oversubscription ratio in the DragonFly+ network design applies to many more traffic patterns than previously proven, as long as hosts don’t exceed full sending or receiving capacity. They also identify some errors in the original proof but confirm the final result remains valid. Furthermore, they find tweaks to network wiring can slightly improve performance under certain conditions.
What this means in practice
- •For data center network designers: Design network topologies with predictable oversubscription levels across a range of traffic patterns to optimize hardware usage and performance.
- •For high performance computing engineers: Select and tune DragonFly+ networks aware of small wiring improvements that yield measurable speedups under specific switch configurations.
Tested on simulated data.
Authors
Vlad-Adrian Ulmeanu, Costin Raiciu, Iulian-Ilie Drăcea
Abstract
The Max-Host Dragonfly+ topology's original paper proves that there is a 2:1 worst-case oversubscription ratio in expectation for the permutation traffic pattern. We show that the proof only covers a subset of permutation patterns, specifically those in which all host pairs are in different groups, and for any receiver group there are at least two sending hosts in distinct groups. The proof contains two mistakes that cancel out to produce the correct result. Furthermore, the proof implicitly uses an important observation without a backing argument: traffic leaving an indirect group, now forced to follow min-cost paths to receiver groups, can still be split across almost all outgoing global links, in contrast with splitting traffic from a sender group that can naturally select any outgoing group. We prove that the 2:1 ratio holds in expectation for a larger set of patterns, including any permutation. We only need to know that any host sends and receives at most line rate traffic. This constraint can be altered to obtain approximations for expected FCT bounds on any pattern. We also attack the oversubscription problem without the expectation assumption, and find bounds that hold with high probability on permutations for small switch radixes. While in expectation the global layer wiring doesn't affect the oversubscription rate, we experimentally find that it matters in the general case. We find topologies that obtain visible speedups against the default global wiring from the original paper, under $7\%$ for radixes at most $8$.