SACQ improves long-term time series forecasting with structured decoding

SACQ: Structured Decoding with Memory-Conditioned Refinement for Long-Horizon Forecasting

Machine Learning

Summary

Long-term forecasting of time-based data often struggles because current models treat all future predictions as linked in a way that hides how specific past moments relate to each future event. The authors introduce SACQ, a new method that first makes a rough forecast and then improves each future point by focusing on the most relevant past information. This approach makes predictions more accurate and resistant to noisy or corrupted data. It works well with existing forecasting models and keeps computational cost low.

What this means in practice

  • For data science teams: Improve robustness and accuracy of long-range forecasts on time series with noisy or corrupted inputs in industrial or financial data.
  • For forecasting software engineers: Integrate SACQ as a plug-in prediction head to enhance existing time series models without retraining encoders.

Authors

Guo Cheng, Zhengzhuo Xu, Chenchen Jing, Jingyi Hou

Abstract

Long-term time series forecasting (LTSF) models predominantly employ patch-based encoders terminated by a flatten readout head that maps the entire encoded historical memory to all future steps through a single shared projection. This implicit coupling of future positions obscures position-specific historical-to-future alignment and amplifies sensitivity to corrupted inputs and extreme supervision noise. We present SACQ, a plug-in structured prediction head that replaces flatten readout while keeping the encoder unchanged. SACQ adopts a two-stage decoding pipeline: it first establishes a coarse patch-grid forecast scaffold, then refines each future position through cross-attention over historical memory and merges the attention-derived correction with the coarse scaffold via a learned per-patch gate. To stabilize optimization under long horizons and noisy labels, we further propose a batch-adaptive scaled log-cosh loss that automatically calibrates robustness to the current residual scale, suppressing outlier gradients while preserving MSE-like sensitivity for typical errors. SACQ attains top-tier test MSE/MAE across PatchTST, DLinear, and patch-Mamba backbones with only modest incremental overhead in parameters and latency. Under inference-time input corruption and training-set label-noise stress tests, SACQ substantially outperforms flatten readouts, with ablation studies validating each architectural component.