Papers for

data center cooling teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

HeatCache schedules LLM inference to save energy and reduce overheating

HeatCache: Thermal-aware Energy-efficient LLM Inference Scheduling for Chassis-level Liquid Cooling in Sustainable Edge Server Rooms

Abstract: LLM inference is increasingly deployed at institution-scale edges to meet service requirements. However, multi-GPU inference consumes a large amount of electricity and produces substantial heat. To improve sustainability, operators and regulations often demand raising the ambient setpoint to reduce cooling electricity. This can increase thermal throttling and hardware aging, leading to Service-Level Objective violations. In this paper, we present HeatCache, a thermal-aware, energy-efficient LLM inference scheduler for commercial chassis-level AIO liquid-cooled GPUs at sustainable ambient temperatures. HeatCache treats AIO loops as a temporary heat buffer, measured by heat budget and schedules requests to minimize energy subject to thermal safety and SLO constraints, based on an electrical-informed heat-demand estimation from HeatiTS. We implement HeatCache atop vLLM and show that it reduces computing energy by up to 18.0%, decreases thermal-throttle exposure by 81.7%, and maintains SLO violation rates below 0.9% even up to $48~^{\circ}\mathrm{C}$.

Fri 11 SeptDistributed, Parallel, and Cluster ComputingPerformance

The gist

Large language models (LLMs) running on multiple GPUs use a lot of electricity and generate heat, which can cause hardware issues if cooling is less efficient. The authors present HeatCache, a new scheduler that carefully plans how requests are run on GPUs cooled by liquid systems to save power and avoid overheating. HeatCache treats the cooling system as a temporary heat buffer to manage workloads without breaking performance rules. Their tests show significant energy savings and much less overheating while keeping service delays low, even at higher ambient temperatures.

Open → 2609.12449v1