Adaptive regularization improves deep learning for time series data
Towards Robust Time Series Learning via Capacity-Centric Modulation
Machine Learning
Summary
Deep learning models for time series data often treat all data points the same during training, which can make the models less reliable when some data points are noisy or corrupted. The authors introduce a method called SACM that adjusts how strongly each data point is regularized based on its characteristics, helping the model focus better on trustworthy information. This approach fits into existing models without changing their design and keeps predictions fast. Tests across many real-world datasets show it improves accuracy and reduces errors in forecasting, classification, and anomaly detection.
What this means in practice
- •For time series engineers: Enhance model accuracy and robustness in time series forecasting pipelines by integrating SACM without changing existing architectures or adding inference cost.
- •For anomaly detection teams: Improve fault or anomaly detection in operational systems using SACM to better handle unreliable or noisy input data during model training.
Authors
Siru Zhong, Senzhang Wang, James T. Kwok, Yuxuan Liang
Abstract
Sample-level reliability heterogeneity is common in deep time series learning. Standard training pipelines apply a uniform regularization setting to all samples, which can under-regularize corrupted samples and over-restrict clean samples. Common robustness approaches filter observations in data space or impose priors on latent representations. We propose Capacity-Centric Modulation (CCM) as a complementary, sample-adaptive regularization principle. Under this principle, we introduce SACM (Sample-Adaptive Capacity Modulation), a task-agnostic framework that exploits spectral sparsity to assign sample-wise dropout probabilities along internal activation paths. SACM integrates into existing backbones without architectural redesign and preserves the deterministic inference pipeline. Across 301 real-world dataset-backbone pairs covering 9 forecasting, 32 classification, and 4 anomaly-detection datasets, SACM reduces forecasting MSE by 6.7% on average and improves classification accuracy and point-adjusted F1 by 3.04% and 17.05%, respectively, relative to unmodified backbones, with zero test-time overhead.