Autoregressive forecasting boosted by separating model roles and speeding execution
One Patch, Three Roles: What Is Actually Coupled in Autoregressive Time-Series Forecasting?
Machine Learning
Summary
Forecasting future values in time-series data often uses a method called autoregressive forecasting where predictions depend on prior steps. This paper shows that different parts of this process—how data is represented, how predictions evolve, and how the model runs recursively—can be changed independently to improve speed and accuracy. The authors introduce a technique called Autoregressive Trajectory Distillation (ATD) that speeds up prediction by reducing repeated calculations while keeping results reliable. They also present a correction method named Spectrum Tangent to further improve forecast accuracy without making the model more complex.
What this means in practice
- •For time-series forecasting teams: Increase forecasting speed by over five times without losing accuracy using Autoregressive Trajectory Distillation in patch-based models.
- •For financial modelers: Improve long-term forecast accuracy through a correction method that refines prediction errors without adding complexity or heavier computations.
Authors
Ziang Li, Yue Huang, Guoxu Zhou, Na Han, Jie Wen, Lunke Fei, Xiaozhao Fang
Abstract
Patch-based autoregressive time-series forecasting often ties input representation, learned transitions, and recursive execution to one patch length. We ask which of these roles can be adjusted separately. A supporting atomic-encoding study finds greater sensitivity to model width than to atom grouping on the evaluated grid. Our main finding is that a frozen parent's recursive trajectory is easier to fit than the observed future with lightweight parallel exits. Autoregressive Trajectory Distillation (ATD) turns this into selectable ATD-1/2/4/8 execution, with ATD-1 exactly recovering the parent. On a paired four-data-set comparison, ATD-8 reaches $5.54\times$ end-to-end speedup with stable quality across widths. Fewer calls do not automatically remove the parent's existing forecast error: ATD improves trajectory fidelity in all 21 seed runs but forecast accuracy in only 15 against matched clean-future supervision. We further find a correctable residual projection along a train-selected periodic history direction. Spectrum Tangent applies this correction without adding neural parameters or Transformer calls. At horizon 720, it reduces mean squared error (MSE) and mean absolute error (MAE) by 2.54% and 2.33% over seven data sets and two output widths, while remaining $3.24\times$ faster than recursive inference. Level and shape projections sometimes disagree. Trajectory compressibility, the fidelity-accuracy mismatch, and the correction recur across three public AR parents. Together these results separate representation, transition, and execution as AR design axes. Code is available at https://github.com/RowanFFF/ATD-Spectrum-Tangent.