Papers for

data center network engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Quantized key value transfer speeds up prompt processing by over two times

ReKV: Overlapping Quantized KV Transfer and Suffix Recomputation

Abstract: Prefill--decode disaggregation moves a prompt's key--value (KV) cache between workers before the first token can be generated. Reducing this handoff to either cache compression or recomputation misses a useful choice: which layers to transmit, at what precision, and which layers to execute again on the decode worker. ReKV transfers a quantized prefix KV cache and one BF16 boundary state, then reconstructs a contiguous suffix while the prefix is in flight. In a model-shaped synthetic benchmark on cross-node H100s, the Q8 configuration meeting separately evaluated quality criteria reduces 8K handoff latency by $2.36\times$, $2.84\times$, and $3.65\times$ over BF16 transfer for Qwen2.5-7B, Qwen2.5-3B, and Qwen3-8B. It is also faster than the best measured transfer-only or recompute-only action at those points. A separate real-weight BF16 recomputation experiment reduces prefill-to-token-return time by $1.16$--$1.63\times$. The preferred action changes with prompt length and link bandwidth.

Thu 5 NovDistributed, Parallel, and Cluster Computing
The gist
Generating text using AI models often requires moving the model’s memory, called the key-value (KV) cache, between computers, which can be slow. The authors present ReKV, a way to send a smaller, compressed version of this memory and only recalculate some parts on the spot, speeding up the overall process. They tested this on powerful Nvidia GPUs and showed it can reduce transfer times by more than twice compared to sending full precision data. This method adapts depending on prompt size and network speed to find the best balance between transfer and recomputation.
Open → 2611.08770v1

OpenRDMA improves GPU communication with efficient multi-path network transport

OpenRDMA: A GPU-Native Multi-Path Reliable RDMA Transport Architecture

Abstract: As the computing power density of large-scale AI clusters rises, network communication increasingly limits system performance. GPU-initiated communication requires low-overhead request submission to overlap communication with computation. However, GPUDirect RDMA (GDR) still relies on CPU involvement in its control path, while InfiniBand GPUDirect Async (IBGDA) requires GPUs to assemble Work Queue Entries (WQEs), consuming additional SM resources and memory bandwidth. We propose OpenRDMA, an open-source multi-path reliable RDMA transport architecture designed for GPU-friendly communication. First, OpenRDMA provides a minimal GPU doorbell mechanism: the GPU issues an 8-byte token, and a dedicated FPGA NIC reconstructs the complete WQE from preconfigured context. Second, building on Multipath Reliable Connection (MRC), OpenRDMA supports per-packet multi-path transport and out-of-order arrivals, while selective retransmission, ordered DMA placement, and GPU-visible flag polling provide reliable delivery and safe data consumption. Third, a hardware-software co-designed retransmission mechanism keeps the GPU submission path lightweight and reduces FPGA hardware state overhead: the NIC handles fast data transfer, while CPU software manages complex timeout detection, state tracking, and recovery policies. We describe the architecture and key hardware components of OpenRDMA; implementation of the FPGA prototype is underway.

Thu 5 NovHardware Architecture
The gist
As AI computers get more powerful, they need to share data quickly between GPUs and other parts. Current methods make the CPU do extra work or slow down the GPU with complex tasks. The authors created OpenRDMA, which lets GPUs send simple signals that a smart network card uses to handle data efficiently over multiple paths. This setup keeps communication fast and reliable without burdening the GPU or CPU too much. Their design is open source and running on specialized hardware is in progress.
Open → 2611.07559v1

Optical circuits improve large language model response speed and energy use

Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode

Abstract: LLM inference decode executes fine-grained collectives whose latency directly limits per-user interactivity, yet scale-up fabrics are still built from electronic packet switches designed for training. We observe that, for a given model deployment and parallelization, the communication structure of decode is fixed, and propose workload-static optical circuit switching (OCS): circuits are configured once per deployment, co-designed with model parallelism and with topology-aware collective schedules that turn spare accelerator ports into fewer communication phases or higher bandwidth. We evaluate this design with GXNET, a graph-based workload emulator coupled to analytical and flit-level network models grounded in hardware measurements. In a 576-accelerator scale-up domain serving 5T-parameter MoE models with up to 5M tokens of context, all-OCS fabrics improve interactivity by up to 65% and energy efficiency by up to 119% over 4:1 oversubscribed packet-switched fabrics.

Tue 3 NovDistributed, Parallel, and Cluster ComputingArtificial IntelligenceHardware Architecture
The gist
Large language model (LLM) decoding needs fast communication between many processors, but current networks slow things down because they were made for training, not for this decoding. The authors found that the communication pattern during decoding stays the same for a given model setup, so they designed a special optical circuit network that is fixed for each deployment to speed things up. This approach uses fewer communication steps and higher bandwidth by cleverly connecting processors with light-based circuits instead of traditional packet switches. Their tests show this design can make LLM responses up to 65% faster and save energy by more than double in large deployments.
Open → 2611.05104v1

Constant factor approximation improves coflow scheduling in parallel networks

Constant-Factor Approximation for Non-Splitting Coflow Scheduling in Heterogeneous Parallel Networks via Iterative Relaxation

Abstract: Coflows capture the collective communication requirements of data-parallel applications, and scheduling them well is essential to the performance of modern data centers. We study non-splitting coflow scheduling in a heterogeneous parallel network whose $m$ network cores may be Electronic Packet Switches (EPS), not-all-stop Optical Circuit Switches (OCS), or all-stop OCS, where every flow must be transmitted entirely through a single core. Previous approximation algorithms for this setting have guarantees that grow linearly with the number of cores $m$ or with the maximum flow degree $τ$ of a port, and therefore become weak as data centers deploy more cores. The source of this dependence is the rounding step: rounding each flow to the core holding its largest fractional value can inflate the load of a port by a factor of $m$. We replace this step with an iterative relaxation rounding scheme that handles the input-port and output-port capacity constraints simultaneously. For minimizing the makespan, this yields approximation ratios of $\min\{τ, 3\}$, $2\min\{τ, 3\}$, and $4\min\{τ, 3\}$ for pure EPS, pure not-all-stop OCS, and pure all-stop OCS networks, respectively. For minimizing the total weighted coflow completion time, we obtain approximation ratios of $8\min\{τ, 4\}$, $16\min\{τ, 4\}$, and $32\min\{τ, 4\}$ for the same three architectures. In a hybrid network, the guarantee is governed by the weakest architecture present, so the overall ratios are at most $4\min\{τ,3\}\le 12$ and $32\min\{τ,4\}\le 128$, respectively. Unlike all previous guarantees for non-splitting coflow scheduling in heterogeneous parallel networks, these bounds are absolute constants that do not grow with the number of cores $m$ or with the flow degree $τ$. We also show that the integrality gap of the underlying linear program is at least $2$.

Thu 29 OctData Structures and Algorithms
The gist
Managing how data moves inside big data centers is tricky because many pieces of information need to be sent together as groups called coflows. The authors found a better way to schedule these groups across different network types without breaking them up, which helps keep data moving smoothly. Their method guarantees steady performance regardless of how many parts the network has, improving over old methods that got worse as networks grew. They use a new mathematical approach that handles network limits more effectively for different setups.
Open → 2610.39564v1

Orchestration platform enables fast deployment of data center network controls

CCAaaS: An Orchestration Platform for High-Velocity Design, Development, and Deployment of Data Center Transports at Meta

Abstract: Today, data center networks run multiple congestion control algorithms (CCAs), each tailored to distinct traffic characteristics, from latency-sensitive intra-region RPCs to high-throughput cross-region AI workloads. The rapid evolution of AI training and inference further accelerates this trend by continually introducing new traffic characteristics that demand new CCAs. This paper introduces CCA-as-a-Service (CCAaaS), our eBPF-based ecosystem for the rapid design, deployment, and operation of production-grade CCAs at hyperscale. We present (1) a modular BPF-TCP framework and general lessons for designing modular CCAs; (2) a deployment and orchestration layer that provides disruption-free CCA updates, along with best practices for managing a zoo of CCAs at scale; and (3) a lightweight observability tool for CCAs and a blueprint for scaling CCA observability. CCAaaS manages over 97% of our server-to-server traffic using three active eBPF-based CCAs; two of the five CCAs we have built have since been deprecated. We show that our eBPF-centric approach to CCAs development delivers the same application-level performance (retransmissions, bandwidth, and tail latency) as their native in-kernel counterparts, and that our custom CCAs add negligible CPU and memory cost over their eBPF baselines. In terms of realized benefits, the CCA built with CCAaaS significantly improved network, and application metrics, for example, one of our CCAs reduced P99 latency for cross-region services by up to 20X.

Fri 23 OctNetworking and Internet Architecture
The gist
Data centers run different ways to manage traffic depending on the type of data being sent, like quick messages or large AI computations. The authors created a system called CCA-as-a-Service (CCAaaS) to quickly develop and update these traffic controls without stopping the data flow. This system uses a technology called eBPF to build flexible, fast, and efficient tools that monitor and manage network congestion. Using CCAaaS, Meta handles most of its server traffic smoothly and can improve network speed and reduce delays by a lot.
Open → 2610.33231v1

Optical networks keep data flowing fast despite failures

OMRC: Multipath Reliability for Direct-Connect Optical Interconnects

Abstract: Network failures degrade Multipath Reliable Connection (MRC) in direct-connect optical datacenters. We propose Optical MRC (OMRC), which remaps collectives and circuits, preserving 95.7% of training throughput, restoring stalled inference, and achieving 4.7x MRC's simulated bandwidth.

Thu 22 OctNetworking and Internet Architecture
The gist
Network failures can slow down or stop data tasks in direct optical connections used in data centers. The authors introduce Optical MRC (OMRC), a method that reroutes these connections when problems occur. This approach helps keep machine learning training fast and restarts halted tasks. It also greatly increases the network's bandwidth compared to previous methods.
Open → 2610.30551v1

Information geometry guides reconfigurable optical networks in AI clusters

An Information-Geometric Framework for OCS-Reconfigurable Interconnect Topology in Large-Scale AI Clusters

Abstract: Optical Circuit Switches (OCS) make inter-pod wiring in AI clusters programmable, posing design questions on wiring choice, symmetry departure, and reconfigurations. We present a unified information-geometric framework to address these issues.A wiring is a point in a probability simplex over circuits within a degree polytope. Departure from a symmetric Fat-Tree prior $\mathbf{q}$ is measured by $\DKL(\cdot \,\|\, \mathbf{q})$. The cost $U$ is the smoothed DAG critical-path length, whose gradient equals path marginals of a Gibbs distribution via forward--backward recursion.For free energy $F_{β_1} = U + β_1^{-1} \DKL(\cdot \,\|\, \mathbf{q})$, we prove well-posedness, monotonic dependence on $β_1$, adaptation bounds via Jeffreys divergence, and a symmetry principle for static optimality. An entropic proximal-gradient method updates multiplicatively with a $|V|$-dim dual KL projection, achieving linear convergence.For time-varying traffic, a threshold policy yields regret $O(\sqrt{ΓC V_1 T})$ against the switching-free optimum. Traffic changes are detected by a multinomial GLR--CUSUM detector with non-asymptotic false-alarm bounds. Multi-job generalized-mean fairness aligns with $α$-fairness ($α\ge 2$). Experiments on pipeline, data, and MoE workloads validate the framework against integer programming.

Sun 18 OctDistributed, Parallel, and Cluster ComputingMachine Learning
The gist
Large AI computing clusters need flexible wiring between parts to handle different tasks efficiently. The authors introduce a mathematical framework that treats wiring choices like probabilities, measuring how much they differ from a balanced starting point. They provide methods to update these connections quickly and adapt to changing data traffic, ensuring efficient communication paths. Their approach also helps balance fairness among multiple jobs running simultaneously, improving overall cluster performance.
Open → 2610.25173v1

Optical network cuts delays in large language model memory sharing

An Optical KV Cache Network for Disaggregated LLM Serving

Abstract: Disaggregated LLM serving moves each request's KV cache from a prefill GPU to a decode GPU before the response begins, and a coding-agent turn often moves the KV of its whole long context. Behind an oversubscribed packet fabric this transfer adds to time to first token (TTFT). An optical circuit switch (OCS) can give every transfer a full-rate circuit for less than an oversubscribed packet fabric costs, but a mechanical OCS needs tens of milliseconds to reconfigure. We hide this delay inside the prefill. A reservation scheduler commits each circuit while the prefill is still computing, so the circuit is ready when the KV is, and each GPU reaches the OCS over its own wavelength-multiplexed (WDM) optics on one fiber pair. In simulations of GPU clusters on real serving traces with measured prefill times on H100 GPUs, this optical KV cache network matches 400G packet fabrics with adaptive routing at 0.55 to 0.78 times the cost of a 2:1 fabric and tolerates the 50 ms reconfiguration delay of a 384-port mechanical OCS.

Tue 13 OctNetworking and Internet Architecture
The gist
Large language models use GPUs to process and generate text, but sharing their large temporary memories between GPUs can cause delays. The authors show how to use a special kind of optical switch to send this data faster and cheaper than traditional network methods. They carefully schedule the optical switch so its slow parts don’t add more waiting time. Their simulations show this method works well, reducing costs and holding up to long switch delays.
Open → 2610.18444v1

Allreduce method cuts communication time on torus networks

SP-Allreduce: Symmetric Tree Schedules for Tori and Beyond

Abstract: Allreduce is a fundamental communication primitive in large-scale distributed training, and torus interconnects remain widely deployed in high-performance systems. We introduce SP-allreduce, a new allreduce algorithm for $d$-dimensional tori that exploits the symmetry of the topology and routes only along shortest paths. We call an allreduce algorithm \emph{fully bandwidth-optimal} if it incurs no bandwidth overhead, no multihop-induced traffic inflation, and achieves perfect per-round load balance. SP-allreduce is based on a family of carefully symmetrized shortest-path trees that guarantees full bandwidth optimality in the $α$--$β$ model. We prove that it uses the minimum possible number of rounds among all fully bandwidth-optimal allreduce algorithms on $d$-dimensional tori, and therefore the minimum running time in this class. In this sense, SP-allreduce refines Bucket allreduce: it preserves Bucket's key strengths while reducing the number of rounds by a factor of two. We further extend our approach to broader symmetric topologies and to communication trees of prescribed depths. Experimentally, the resulting algorithms improve over the best compared state-of-the-art algorithms for tori by up to $1.19\times$ on square tori and $1.65\times$ on rectangular tori across the tested configurations. On HyperX and spine-leaf networks, we demonstrate gains of up to $14.82\times$ and $1.72\times$, respectively, over the best evaluated baselines.

Mon 12 OctDistributed, Parallel, and Cluster Computing
The gist
Large computer systems often need to quickly combine data from many parts, and the way their connections are arranged affects the speed of this process. The authors created a new method called SP-allreduce that uses the symmetrical structure of torus networks more efficiently. Their approach reduces the number of communication steps needed, making data combination faster, and they also extended it to other network shapes. They tested their method on different networks and found it performs better than existing methods by a noticeable margin.
Open → 2610.16387v1

IAPRepair improves erasure-coded storage repair speed with in-network aggregation

IAPRepair: In-Network Aggregation Enhanced Proactive Repair for Erasure-Coded Storage System

Abstract: Erasure-coded storage provides fault tolerance with substantially lower storage overhead than full replication, but repairing lost or at-risk blocks requires intensive cross-node data transfer. Existing reactive repair schemes start only after a failure, while proactive schemes can move data before failure but commonly treat migration, reconstruction, and network aggregation as loosely coupled operations. As a result, receiver?side bottlenecks, heterogeneous available bandwidth, and limited programmable-switch state continue to constrain repair paral?lelism. This paper presents IAPRepair, an in-network aggre?gation enhanced proactive repair framework for erasure-coded storage. IAPRepair jointly constructs each repair batch, assigns reconstruction providers and replacement nodes according to normalized transmission loads, schedules migration around the remaining receive capacity, and selectively enables in-network aggregation for reconstruction blocks that would otherwise over?load healthy nodes. The selective design reduces receiver-side traffic while retaining migration parallelism and respecting a configurable switch-resource budget. We implement IAPRepair with a Tofino programmable switch and 16 storage nodes, and evaluate it using both a prototype testbed and large-scale simu?lations. Across coding parameters, block sizes, node populations, and multiple STF-node scenarios, IAPRepair reduces repair time by at least 47.83% compared with the evaluated state-of-the-art methods. The results demonstrate that coordinating proactive repair decisions with in-network processing is an effective way to improve repair efficiency under bandwidth heterogeneity.

Thu 8 OctDistributed, Parallel, and Cluster Computing
The gist
Erasure-coded storage keeps data safe with less extra space than full copies, but fixing lost data pieces needs lots of data moving between computers. Previous methods either wait for failures or separate tasks like moving and rebuilding data, causing delays. The authors created IAPRepair, a system that plans repairs ahead, balances data loads, and uses special network switches to combine data during repair. This approach reduces network traffic and speeds up repair times by nearly half compared to current methods. Their tests show it works well across various setups and network conditions.
Open → 2610.11114v1

Optimal data conversion bandwidth found for special fault tolerant codes

Optimal Conversion Bandwidth for MDS Convertible Codes in the Split Regime with $r^F<k^F<r^I$

Abstract: Erasure codes are widely used in distributed storage systems to provide fault tolerance. An $[n,k]$ erasure code encodes $k$ data symbols into $n$ coded symbols and distributes them across $n$ storage nodes. Once the code parameters are fixed, the achievable fault tolerance is also fixed. However, the failure rates of storage nodes may vary over time, and dynamically adapting the code parameters to these variations can substantially reduce storage overhead. Motivated by this observation, Maturana and Rashmi introduced convertible codes, which allow an $[n^I,k^I]$ initial code with redundancy $r^I=n^I-k^I$ to be transformed into an $[n^F,k^F]$ final code with redundancy $r^F=n^F-k^F$, while preserving the required code properties. Convertible codes whose initial and final codes are both MDS codes are called MDS convertible codes. They are of particular interest because MDS codes provide the maximum erasure tolerance for a given amount of storage overhead. Several works have established lower bounds and constructions for the bandwidth cost of MDS convertible codes in the split regime. However, the tight bound in the parameter range $r^F<k^F<r^I$ remains unknown. In this work, we establish a family of new rank inequalities for stable MDS convertible codes with linear conversion procedures and derive an improved lower bound on the bandwidth cost consisting of three cases for this remaining range. We prove that the bound is tight by presenting three explicit constructions, one for each case.

Tue 6 OctInformation Theory
The gist
Distributed storage systems protect data by splitting and encoding it across multiple nodes. Sometimes, the system needs to change how much extra data it stores to handle different failure conditions. The paper focuses on a special kind of code, called MDS convertible codes, which can switch between configurations while keeping data safe. The authors solved an open problem about the best possible bandwidth needed to switch these codes in a tricky parameter range, and they also provide exact ways to achieve this efficiency.
Open → 2610.08349v1

Ofan improves AI training speed by balancing network load better

Ofan: Optimal Load Balancing for AI Training

Abstract: The extreme collective completion time (CCT) demands of AI workloads challenge existing packet spraying algorithms, which can have trouble efficiently load-balancing workloads that are sent at full line rates. We trace this to a structural cause: on a fat tree, once a packet picks its upward path, the downward path to its destination is unique, so destination-oblivious schemes cannot undo the imbalance it creates. We prove that such schemes can suffer from $Θ(\sqrt{m})$ queueing for messages of size $m$, thus eventually triggering rate reductions by the congestion control. Instead, we suggest Ofan, a switch-based destination-aware LB scheme that can reach $O(1)$ queueing. We also present its pOfan variant that fits the pipe-based architecture of current switches. Our P4 implementation shows that it consumes modest resources. An end-to-end FSDP2 evaluation with Llama-3 405B-parameter models shows that Ofan cuts CCT inflation by $16$--$39\times$ when compared to existing algorithms.

Mon 5 OctNetworking and Internet ArchitectureDistributed, Parallel, and Cluster ComputingPerformance
The gist
Training large AI models requires fast and balanced data movement across network switches. Existing methods can cause traffic jams that slow down training. The authors show that current algorithms have fundamental limits in handling such traffic efficiently on fat tree networks. They introduce Ofan, a new way for network switches to be smart about where data goes, greatly reducing delays. Testing with large AI models shows Ofan can make training run much faster compared to previous methods.
Open → 2610.07230v1

VarioPath speeds up GPU cluster data sharing by managing PCIe traffic

VarioPath: Workload-Aware All-to-All Communication for PCIe GPU Clusters

Abstract: AlltoAllv communication is a critical primitive in distributed large-model inference, particularly for mixture-of-experts (MoE) models. The growing adoption of PCIe GPU systems for cost-efficient inference makes AlltoAllv performance on these systems increasingly important. Without a dedicated scale-up interconnect (e.g., NVLink or Infinity Fabric), PCIe GPU systems carry both intra-node and inter-node traffic through the PCIe hierarchy, where concurrent transfers can contend for PCIe link bandwidth. This link contention, compounded by skewed traffic distributions and dynamic traffic demand, makes efficient AlltoAllv scheduling challenging. Existing approaches are either poorly suited to PCIe GPU systems or incur substantial schedule synthesis overhead that reduces their practicality in real-world deployments. We present VarioPath, an efficient AlltoAllv scheduling framework for PCIe GPU systems. It combines an offline topology-aware analyzer with an online demand-aware scheduler. The analyzer records contention-free transfer patterns as AlltoAllv channels and exploits topology symmetry to build a compact catalog for efficient search. The online scheduler decomposes each AlltoAllv invocation's demand across a sequence of channels, adapting to rapidly changing and skewed traffic while incurring low planning overhead. Evaluation on four platforms (up to 256 GPUs) shows average AlltoAllv speedups of 5.88x over FAST and 1.72x over DeepEP. End-to-end experiments show that VarioPath reduces Qwen3 inference latency by up to 27.2% and Wan2.1 generation latency by 6.1%.

Mon 28 SeptDistributed, Parallel, and Cluster ComputingNetworking and Internet Architecture
The gist
Sharing information quickly between many GPUs is important for running large AI language models smoothly. On systems using PCIe connections, the data transfer can get jammed because many transfers compete for the same routes. The authors designed VarioPath, a method that smartly plans these data transfers based on the computer setup and current demands. This approach helps reduce delays and speeds up tasks like AI model inference on GPU clusters.
Open → 2609.34340v1

Flux schedules optical switches to speed up AI model training

Flux: Optimal Scheduling of Optical Circuit Switches for LLM Training

Abstract: Optical Circuit Switching (OCS) offers high bandwidth density and energy efficiency for LLM training, but incurs a non-negligible reconfiguration delay. Prior work typically schedules optical circuit switches independently of compute, using aggregate traffic demand to determine which circuits to provision and when. We argue that this separation creates a fundamental inefficiency: reconfigurations that ignore the compute timeline can stall communication, resulting in low circuit utilization and large buffer requirements. In this paper, we present Flux, a scheduler that optimally schedules optical circuit switches based on the structure of the entire workload. Flux remains effective across a wide range of switching speeds by reusing circuits and amortizing reconfiguration delay behind compute and communication. We show that Flux reduces training iteration time by up to $10\times$ and peak NIC buffer requirements by more than three orders of magnitude compared to traditional periodic schedulers.

Tue 22 SeptNetworking and Internet ArchitectureDistributed, Parallel, and Cluster Computing
The gist
Training large AI models needs fast and efficient data movement between computers. Optical circuit switches offer great speed but take some time to change connections, causing delays. The authors argue that planning switch changes without considering the training steps slows things down and wastes resources. They created Flux, a scheduler that plans switch changes based on the whole training work, reducing delays and buffer needs. Flux can make training up to ten times faster and greatly reduce memory use compared to older methods.
Open → 2609.25949v1

Fpga design tracks packets fast for reliable wide area networks

Scalable Packet Tracking on FPGAs for Erasure-Coded RDMA over Lossy WANs

Abstract: Modern AI workloads increasingly rely on scale across architectures that interconnect multiple datacenters to form a single "AI factory", overcoming the power and cooling constraints of individual sites. However, extending Remote Direct Memory Access (RDMA) across wide area networks (WANs) introduces fundamental challenges: multi-path packet reordering, high latency, and packet loss that severely degrade performance. While erasure coding (EC) has emerged as a promising mechanism for loss recovery, its effectiveness critically depends on efficient packet arrival tracking implemented in hardware. We present COmpact Multi-path Erasure-coded Tracking (COMET), the first fully hardware-offloaded packet-arrival tracking design implemented on an FPGA-based network interface card (NIC) for multi-path RDMA over lossy WANs. COMET employs a scalable cache-based architecture that supports operation at high link rates. Our evaluation shows that COMET sustains line rate operation at 400 Gbps and beyond. Critically, COMET decouples on-chip memory footprint from link Bandwidth-Delay Product (BDP), and its cache-based architecture (COMET Cache) enables supporting 6 times more concurrent connections than state-of-the-art (SOTA) SoC-based designs. These results demonstrate that scalable, fully hardware-offloaded packet-arrival tracking is practical on FPGA-based NICs at current data rates, and its architectural scalability extends to emerging 1.6 Tbps NICs and beyond.

Fri 18 SeptHardware Architecture
The gist
Connecting multiple data centers to work together for AI tasks is tricky because data sent over long distances can get mixed up, delayed, or lost. The authors show that a special computer chip called an FPGA can help keep track of data packets quickly and efficiently, even when some are lost along the way. Their design, called COMET, can handle very fast data connections without slowing down, making it easier to send data reliably over large networks. This can help data centers work better together at huge scales.
Open → 2609.21774v1

How words turn into network data during large AI model training

The Life of a Token: from Words to Bits on the Wire

Abstract: Large Language Models (LLMs) transform vast collections of unstructured text into semantic patterns used for language generation and reasoning tasks. Behind their ease of use lies a complex process: words become tokens, tokens become vectors, and vectors ultimately give rise to streams of bits that flow through High-Performance Computing (HPC) systems. As modern LLMs grow to billions or trillions of parameters, this path increasingly unfolds across thousands of interconnected accelerators, making the underlying communication fabric a critical and often opaque component of model training. This tutorial aims to walk the reader through the journey from words to network traffic, shedding light on how language is translated into communication flows within HPC training systems. Using concrete examples from Dante's Divine Comedy, we illustrate how model architecture, tokenization, embeddings, and parallelization strategies shape the volume, structure, and timing of data exchanged across the network. We combine architectural analysis with analytical traffic models and numerical examples to characterize the communication requirements of LLM training. We try to demystify how words travel across the network and provide practical insights into the network requirements needed to support the journey from text to trained model.

Thu 17 SeptDistributed, Parallel, and Cluster ComputingMachine LearningNetworking and Internet Architecture
The gist
Training huge AI language models involves turning words into bits that travel across many computers working together. The authors explain the detailed process of how words are broken down, turned into numbers, and then sent as data across networks in high-performance computing systems. They use examples from a famous book to show how different parts of the model and training setup affect the communication between computers. This helps people understand the network needs for training large language models.
Open → 2609.19924v1

Low-latency flow-based intrusion detection for FPGA smart network cards

FSNIC: A Low-Latency Flow-Based Intrusion Detection Architecture for FPGA SmartNICs

Abstract: Modern data centres require high-performance networking alongside effective real-time security. Traditional Intrusion Detection Systems (IDS) commonly rely on general-purpose processors and often struggle to inspect high-speed traffic at line rate without introducing latency or performance bottlenecks. Smart Network Interface Cards (NICs) provide an alternative by enabling computation directly within the network data plane. This work presents a machine learning-based IDS implemented within an FPGA-based SmartNIC pipeline. The system integrates P4-based packet parsing with a LogicNets IDS model implemented in RTL, enabling deterministic, low-latency inference. Compared with traditional stateless packet-level classifiers, the proposed stateful flow-based IDS introduces minimal state by aggregating features across packets, capturing behavioural patterns not observable at the packet level. Experimental results on the UNSW-NB15 dataset show that the flow-based IDS improves detection accuracy from 86.92\% to 97.68\% compared with stateless packet-level classification. We also evaluate the proposed IDS on CICIDS2017 and compare its real-time hardware performance with prior FPGA-based IDS designs. Through hardware-software co-design, the proposed IDS achieves 6~ns inference latency using only 846 LUTs, with no BRAM or DSP usage, demonstrating a low latency and resource efficient implementation.

Mon 14 SeptHardware Architecture
The gist
Data centers need strong and fast security systems to detect network attacks without slowing down traffic. The authors designed a machine learning system that runs directly on smart network cards with FPGAs, allowing quick and efficient recognition of attacks by looking at sequences of packets rather than single packets alone. This approach improves the accuracy of detecting threats and processes data with very low delay and minimal hardware resources. Their tests showed better detection rates and faster performance compared to earlier methods.
Open → 2609.16363v1

Optical switch scheduling boosts large language model training speed

Breaking the Duplex Barrier: Lane-Granularity OCS Scheduling for LLM Training

Abstract: Optical circuit switch (OCS) can reconfigure physical connectivity to match the predictable communication schedules of large language model (LLM) training. Although each OCS light path is physically simplex, existing demand-aware OCS schedulers allocate capacity in duplex-port pairs, forcing equal bandwidth in both directions and stranding capacity under asymmetric node-pair traffic. This paper present LACE, the first offline OCS schedule compiler that independently allocates transmit (TX) and receive (RX) lanes for LLM training. Without changing the selected collective algorithms, operation order, or rank placement, LACE reconstructs directed node-level demand, jointly determines which consecutive operations share a configuration and how many simplex circuits serve each direction, and realizes these allocations as physical lane bindings and optical paths under per-node lane-inventory and multi-OCS fabric constraints. Software acknowledgments carry feedback over independently provisioned return paths, while coordinated link configuration and recovery verify each configuration before communication resumes. On a separate three-server testbed using fixed topologies and matched per-port rate limits, LACE's asymmetric connectivity achieves $1.80\times$ speedup for communication replay and $1.27\times$ for GPT-2 training over a symmetric-topology baseline. At larger scale, simulations of LLaMA-3.1 70B and 405B schedules with sixteen 400-Gb/s ports per server show that LACE achieves $1.21$--$2.04\times$ communication speedup over the latest duplex OCS scheduler.

Sun 13 SeptNetworking and Internet Architecture
The gist
Large language models need lots of data to be sent between computers, but current optical switches limit data flow by treating connections as two-way equal paths. The paper introduces LACE, a system that breaks this limit by allowing one-way (simplex) data lanes, adapting to uneven communication needs. This approach speeds up data transfer during training without changing the overall training setup. Tests show it can make training faster by up to nearly twice as much in some cases.
Open → 2609.14253v1