AI data center planning faces challenges in tracking decision impacts
Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout
Artificial Intelligence
Summary
Planning an AI data center involves many early decisions that determine its energy use and emissions, but current language models struggle to understand all the consequences of these choices. The authors created DCBench-440, a new test with many questions about real and simulated data center planning scenarios, to see how well AI can reason about these decisions. They found that models often miss important actions and fail to list all possible choices, showing limits in AI reasoning for this area. This benchmark helps measure progress in teaching AI to better understand complex planning commitments.
What this means in practice
- •For data center planners: Evaluate AI tools’ ability to predict how early infrastructure decisions affect energy and emissions over time.
- •For environmental compliance teams: Screen AI model outputs for missing or incorrect reasoning about regulatory and environmental implications in facility buildouts.
Authors
Pankaj Chouhan, Rajarshi Das, Aditya Dubey, Manminder Tomar, Sreekar Reddy, Samta Sharma, Satyapragnya Kar, PR Panigrahi
Abstract
An AI data center is committed years before it draws a watt: a few discrete choices (grid or on-site gas, evaporative or dry cooling, which county to file in) fix its capital, water draw, and carbon path, and LLMs are now asked to help screen them with no test of whether they track what each commitment rules out. We introduce DCBench-440, with two arms reported separately: 200 generative questions over STRIPS worlds for five US deal archetypes, with search-verified gold answers and no tested shortcut above 12% on any question type, and 240 multiple-choice questions grounded in quoted site records, emission equations, and stated scenarios. Across open-weight deployments, accuracy spans 12.5-71.5% (generative) and 58.3-92.5% (multiple choice). One skill fails everywhere: asked to list every action available at a decision gate, no model exceeds 4.0% exact match. The strongest list 72-85% of the actions that would change the project's state but omit most permitted actions that change nothing; with those excused, the best exact match is still 24%. Code, data, and other materials are available upon request.