Variable-wise optimization improves multivariate time-series forecasting accuracy
Not All Variables Agree: Reliability-Aware Variable-Wise Gradient Surgery for Multivariate Time-Series Forecasting
Machine Learning
Summary
When forecasting multiple related time measurements, combining errors into one average can hide important details. The authors found that the directions of individual variable contributions often disagree during training, and this disagreement isn’t always linked to worse performance. They propose a new way to separate and adjust variable-specific training signals, leading to better forecasts across different models and datasets. Their method handles each variable’s impact more carefully, improving accuracy by a few percent.
What this means in practice
- •For data engineering teams: Enhance forecasting models for multiple measurements by improving gradient handling during training to boost accuracy without changing model architecture.
- •For software developers building iot systems: Improve sensor data forecasting accuracy in IoT devices by applying variable-wise gradient adjustments during model training.
Authors
Jinwoo Park, Hyeongwon Kang, Pilsung Kang
Abstract
In data-driven training, multivariate time-series forecasting is usually optimized with a scalar loss averaged over samples, variables, and horizons. This averaging is convenient, but the optimizer sees only the aggregated gradient, which does not reveal whether the variable-wise contributions align or oppose one another. To quantify how often this disagreement arises, we measure the variable-wise gradients directly and find that 30.6% of their pairwise cosine similarities are negative on average across seven datasets. However, conflict and harm are not the same thing. Under shared training 35 of the 64 variables do worse than a full-input single-target oracle, and the harmed fraction is not reliably predicted by how often gradients conflict. We propose Per-Variable Surgery (PV-Surgery), an optimizer-side training strategy for backbones with cache-compatible layers. One backward pass builds variable-wise gradient proxies from output-side signals and keeps the pointwise forecasting loss. Reliability-aware selection targets layers whose proxy sums closely approximate their shared-gradient slices. Conditional pooling forms anchor and conflict pools without dropping variables. Common-direction surgery aligns variable or pooled gradients with their normalized mean and restores input norms to avoid reweighting. In experiments across five backbones, seven datasets, and four horizons, PV-Surgery lowers MSE by 3.61% and MAE by 2.93% on average. For multivariate forecasting, this indicates that the variable-wise structure hidden by mean-loss training is a usable optimization signal.