Papers for

data center managers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Distributed resource allocation using single-bit gradient communication

Low-Bit Distributed Constrained Optimization with Application to Optimal Scheduling and Allocation

Abstract: This paper proposes a distributed resource allocation algorithm in which each agent communicates only the sign of its local cost gradient, a single bit per directed link per iteration. The algorithm is based on Laplacian-gradient dynamics and preserves the equality (demand) constraint at every iterate, guaranteeing anytime feasibility irrespective of termination time. First, we provide a complete Lyapunov-based convergence analysis for the fixed-step-size regime: the iterates are shown to enter, in at most O(1/\varepsilon^2) iterations, a neighborhood of the optimizer whose radius scales as O(τ) in the decision variable and O(τ^2) in the cost residual, where $τ$ is the step-size. Crucially, the step-size bound is a time-invariant network constant (depending only on the graph Laplacian spectrum and the cost curvature bounds), so no adaptive tuning is required. Second, we characterize the invariant set under B-connected, weight-balanced digraphs, strictly weaker than the all-time connectivity and doubly-stochastic weights assumed in most prior work, and show that any odd, sign-preserving nonlinearity yields the same invariant-set and descent structure, with the single-bit protocol being the extreme communication-minimizing instance of a broader nonlinear class. Third, we address local box constraints via smooth penalty relaxation and quantify explicitly how the penalty weight interacts with the step-size bound and the optimality gap. Simulations on economic dispatch (1000 generators) and CPU scheduling (500 servers) demonstrate that the single-bit protocol achieves 5-60 times reduction in total communication cost relative to multi-bit baselines, at the price of a tunable and explicitly bounded optimality gap. A diminishing step-size schedule is shown empirically to eliminate this gap; its formal convergence analysis is identified as an open problem.

Sat 17 OctDistributed, Parallel, and Cluster ComputingMultiagent Systems
The gist
This paper tackles the problem of how multiple agents can work together to allocate resources efficiently while communicating very little information. The authors propose a method where each agent only sends one bit of information per connection at each step, specifically the direction (sign) of their cost gradient. This approach guarantees that the total demand is always met and converges steadily towards a good solution without needing complex tuning. They show that their method greatly cuts communication costs in tasks like power distribution and server scheduling, trading off a small, controllable loss in optimality.
Open → 2610.24431v1

Fpga smartnic speeds scientific data flow monitoring in cloud networks

High Speed Network Processing for Scientific Computing in the Open Cloud Testbed

Abstract: This paper presents an FPGA-based SmartNIC architecture designed to address critical network challenges in data-intensive scientific computing. As scientific workflows increasingly demand high-bandwidth data movement and real-time monitoring across wide-area networks, the Open Cloud Testbed (OCT) provides an ideal platform for investigating in-network processing solutions at line rate. Our approach focuses on implementing network visibility, optimization, and orchestration capabilities through FPGA acceleration. The proposed system leverages P4-programmable data-plane functions to support packet parsing, marking, telemetry extraction, traffic steering, and machine learning inference, all without requiring host CPU intervention or modifications to application servers. As a concrete example, our framework integrates the SciTags standard, which enables associating network traffic with scientific context through packet marking. We implement a high-performance network bridge that counts packets by their SciTag values, as identified in the IPv6 Flow Label. Experimental evaluation on AMD U280 FPGAs in OCT demonstrates the feasibility of this approach, showing imperceptible processing latency and line-rate packet handling, while accurately monitoring scientific data flows.

Mon 5 OctNetworking and Internet ArchitectureDistributed, Parallel, and Cluster Computing
The gist
Moving huge amounts of scientific data across large networks is challenging, especially when monitoring and managing that traffic in real time. The authors present a specialized hardware design using FPGA-based SmartNICs that can process network data very quickly without slowing down the system. Their system can read and tag packets with scientific information, count them, and even run simple machine learning tasks without needing help from the computer's main processor. Tests show their solution works fast enough to keep up with the network speed while accurately tracking scientific data flows in the cloud.
Open → 2610.07377v1

Optical sensor placement cuts data network failures and downtime

Sparse Distributed Fiber Optic Sensor Placement in Metropolitan and Datacenter Elastic Optical Networks

Abstract: Distributed Fiber Optical Sensing (DFOS) leverages optical backscattering signals to predict failure events and enable proactive Just-In-Time (JIT) restoration. Modern network infrastructures have optical components and fibers as key enablers of high throughput and low latency. The integration of sensing features in this optical infrastructure adds features such as network diagnosis, spatial positioning and sensing, and opens up opportunities for the implementation of intelligent network control expected for the next-generation networks. In this paper, we study how the quantity and the placement of the DFOS devices in an all-optical network can mitigate failure events. In our evaluation, we consider metropolitan and data center elastic optical networks. This work also evaluates sensing devices with various prediction time capabilities, in order to assess their effectiveness when considering a variable number of deployed devices and different placement heuristics. The evaluation uses key performance metrics, including the bandwidth blocking rate, number of affected circuits by failure events, and the service downtime. Simulation results demonstrate that the Greedy Shortest Path Coverage (GSPC) heuristic substantially reduces the number of circuits affected by failure when DFOS devices cover only 5% of links, while 25% coverage shows a performance gain comparable to the full coverage scenario, which is an opportunity for significant cost reduction. Finally, for the 5% sensing coverage scenarios, our results illustrate that prediction windows of 15 ms and 7.5 ms are sufficient for JIT restoration in metropolitan and data center networks, respectively.

Fri 2 OctNetworking and Internet Architecture
The gist
When fiber optic networks fail, it can cause big problems for the internet and data centers. The paper shows that by carefully putting a few special sensors that predict failures in the right spots, the network can react faster and reduce problems. The authors tested different strategies in simulations for city and data center networks and found that even covering 5% of network links with sensors helps a lot. This can save money because only a small number of sensors are needed for strong protection.
Open → 2610.03338v1

Trace improves cable tracking accuracy amid clutter and crossings

TRACE: Interactive Bi-Directional Tracing of Monochrome Cables Amid Clutter

Abstract: Accurate state estimation (tracing) of Deformable Linear Objects (DLOs) such as cables is a critical challenge for data centers, manufacturing, construction, homes, and surgery, where precise cable management directly impacts operational safety and efficiency. However, resolving the state of multiple monochrome cables amid foreground and background clutter poses challenges due to occlusions, overlap, and ambiguous crossings. We present Two-way Routing And Cable Estimation (TRACE), which combines bi-directional cable tracing with interactive perception primitives-Divergence Push and Cluster Dilation-to actively resolve ambiguities. Evaluation with 110 physical experiments suggests that TRACE can increase the percentage of cable length correctly traced in complex scenarios (with up to 4 cables and 40 crossings) from ~60% with the strongest prior method, HANDLOOM 2.0, to ~90%, outperforming RT-DLO, Nano Banana Pro, and ChatGPT 5.2 as well. For a trial run on a workstation with an NVIDIA GeForce RTX 4090 GPU, the average computation time is 0.4 seconds per cable. Project website: https://trace-paper.github.io/.

Thu 24 SeptRobotics
The gist
Tracing cables that overlap or cross each other is tricky, especially when they look the same color and get tangled. The authors created TRACE, a method that traces cables in two directions and uses interactive steps to clear up confusion. This approach tracks cables more accurately than previous methods, even in difficult setups with many cables and crossings. It works fast, taking less than half a second per cable on a powerful computer.
Open → 2609.29103v1

Interactive 3d digital twin helps visualize supercomputer hardware and activity

Object Model Analysis of a Supercomputer with Digital Twin

Abstract: Operators and developers need a mental model of both the structure and the live behavior of a large supercomputer, but its physical layout, logical organization, and streams of per-node telemetry are difficult to relate to one another, making it hard to trace a metric or event back to a specific hardware component. We present DAT, an interactive three-dimensional digital analytics twin of a compute cluster built in a real-time game engine, Unreal Engine. DAT expands a compact, parametric description of a supercomputer, a reusable Digital Twin Prototype (DTP), into a navigable Digital Twin Instance (DTI) that mirrors its physical containment hierarchy of racks, chassis, blades, and network links, encoding each node's role and health in its appearance, while a lightweight event-driven simulator animates job and hardware activity over a virtual clock. Our current implementation adds a two-path node-selection mechanism, unifying direct 3D pointing with command-shell queries, that opens an in-world visual-analytics panel beside any selected component showing summary statistics and live, time-varying metrics. We describe this architecture, report qualitative behavior from the working prototype, and outline the path toward driving the panels with recorded telemetry and in-situ anomaly detection.

Fri 11 SeptDistributed, Parallel, and Cluster Computing
The gist
Supercomputers are complex machines made of many parts, and it can be hard to understand how all these parts work together or spot problems. The authors created a 3D digital twin, a virtual model, that shows the supercomputer’s physical setup like racks and network links, along with live activity and health information. This model runs in a game engine and lets users explore and select parts either by clicking in 3D or using commands, showing detailed stats and ongoing metrics. This helps operators and developers quickly connect performance or problems to the right hardware piece.
Open → 2609.13571v1

Mathematical model improves scheduling for large scientific workflows

Mathematical Modeling of a Cognitive Continuum Digital Shadow for Large-Scale, Cross-Facility Workflows

Abstract: We present the mathematical foundations of a \emph{Cognitive Continuum Digital Shadow} (CCDS), a decision-support layer between users and the cross-facility infrastructure---instruments, networks, data stores and compute centers---of exascale and post-exascale scientific workflows. The CCDS couples a state-space representation of the continuum with multistage stochastic programming, so that deployment scenarios can be explored and optimized \emph{before} jobs are launched. This allows operators and users to quantify the cost, makespan and energy trade-offs of a workflow under uncertain resource availability, and hedge their decisions accordingly. We formulate the underlying optimization as a multimode, resource-constrained, stochastic supply-chain network design problem and demonstrate it on a realistic genomics workflow scheduled across heterogeneous HPC and data-center resources. This is the first of three papers; the second treats the underlying software architecture and the third reports large-scale use-cases.

Mon 7 SeptPerformance
The gist
Large scientific workflows often run across many computing centers and instruments, making it hard to plan and manage resources efficiently. The authors present a mathematical approach called the Cognitive Continuum Digital Shadow (CCDS) that helps users and operators understand and predict trade-offs like cost, time, and energy before starting jobs. This approach uses advanced optimization to handle uncertain availability of resources, allowing better decisions on where and when to run parts of a workflow. They demonstrated this method on a genome analysis example using diverse computing resources.
Open → 2609.07275v1