Information geometry guides reconfigurable optical networks in AI clusters
An Information-Geometric Framework for OCS-Reconfigurable Interconnect Topology in Large-Scale AI Clusters
Distributed, Parallel, and Cluster ComputingMachine Learning
Summary
Large AI computing clusters need flexible wiring between parts to handle different tasks efficiently. The authors introduce a mathematical framework that treats wiring choices like probabilities, measuring how much they differ from a balanced starting point. They provide methods to update these connections quickly and adapt to changing data traffic, ensuring efficient communication paths. Their approach also helps balance fairness among multiple jobs running simultaneously, improving overall cluster performance.
What this means in practice
- •For data center network engineers: Optimize programmable optical wiring in AI clusters to reduce communication delays and adapt automatically to workload changes.
- •For cloud infrastructure operators: Detect shifts in cluster traffic patterns promptly to switch network configurations with minimal performance loss.
Authors
Shintaro Yoshizawa
Abstract
Optical Circuit Switches (OCS) make inter-pod wiring in AI clusters programmable, posing design questions on wiring choice, symmetry departure, and reconfigurations. We present a unified information-geometric framework to address these issues.A wiring is a point in a probability simplex over circuits within a degree polytope. Departure from a symmetric Fat-Tree prior $\mathbf{q}$ is measured by $\DKL(\cdot \,\|\, \mathbf{q})$. The cost $U$ is the smoothed DAG critical-path length, whose gradient equals path marginals of a Gibbs distribution via forward--backward recursion.For free energy $F_{β_1} = U + β_1^{-1} \DKL(\cdot \,\|\, \mathbf{q})$, we prove well-posedness, monotonic dependence on $β_1$, adaptation bounds via Jeffreys divergence, and a symmetry principle for static optimality. An entropic proximal-gradient method updates multiplicatively with a $|V|$-dim dual KL projection, achieving linear convergence.For time-varying traffic, a threshold policy yields regret $O(\sqrt{ΓC V_1 T})$ against the switching-free optimum. Traffic changes are detected by a multinomial GLR--CUSUM detector with non-asymptotic false-alarm bounds. Multi-job generalized-mean fairness aligns with $α$-fairness ($α\ge 2$). Experiments on pipeline, data, and MoE workloads validate the framework against integer programming.