Data repair improves AI demand forecasts for US building electricity
Validated Data Onboarding for AI Demand Forecasting on U.S. Building Meter Data: Design, Controlled Evaluation, and a Corrected Negative Result
Machine Learning
Summary
Electric meters in buildings often report faulty data that can confuse AI models predicting electricity use the next day. The authors built a system that finds and fixes these errors using only data available at forecast time. They tested it on real electricity data from twelve buildings and showed that fixing errors greatly improved prediction accuracy. They also found that their first version sometimes removed too much data, making forecasts worse, and shared how they fixed it. All results can be checked and repeated with public data and code.
What this means in practice
- •For utility data teams: Improve next-day electricity demand forecasts by detecting and repairing faulty meter data before training models.
- •For energy management companies: Use validated data-onboarding methods to ensure AI models for building energy load forecasting remain accurate despite sensor errors.
Authors
Yixuan Liang
Abstract
Electric utilities and grid operators increasingly rely on machine-learning models to forecast next-day demand, and those models learn from meter data that is routinely defective: readings go missing, sensors freeze, buildings read zero for hours, and units change by a factor of 100. This report presents a data-onboarding pipeline that detects and repairs such defects before a model is trained, using only information available at forecast time, and a controlled experiment that measures whether the pipeline protects a 24-hour-ahead forecast. On hourly electricity data for twelve U.S. buildings from the public Building Data Genome 2 dataset (210,528 rows, 2016-2017), seeded, hash-logged defects touching 0.10% of the training period raised the error of a gradient-boosting forecaster by 86%; after detection and past-only repair the error returned to the clean-data level (mean absolute scaled error 0.760 clean, 1.415 corrupted, 0.729 repaired) while 93% of training targets were retained. At a defect prevalence calibrated to published field studies (1.6% of training rows) the unprotected forecaster's error reached 4.4 times that of a seasonal-naive rule, and the repaired forecaster again matched the clean baseline. The same pattern held for ridge regression and a random forest and across horizons of 1 to 24 hours. A first version of the pipeline over-cleaned natural data and made forecasts 25% worse; that result is retained, its cause is traced in the published artifacts, and the per-building calibration that corrects it is documented as a dated amendment. Every number is reproducible from pinned public inputs with SHA-256 verification, 84 automated tests and continuous integration.