Time-series models show limits in predicting forced system responses accurately
Attenuated in-context identification in time-series foundation models: diagnosis under counterfactual inputs and repair by synthetic forced-system fine-tuning
Machine Learning
Summary
Time-series foundation models aim to predict how changes in input affect outputs without retraining. The authors found that some models, like TimesFM-2.5, only consider immediate input changes and ignore dynamics, while Chronos-2 partially captures dynamics but underestimates their effects. They showed that adding small input variations or fine-tuning on synthetic data improves response accuracy. However, traditional methods still outperform these models on real measured systems, and fine-tuning slightly reduces forecasting ability.
What this means in practice
- •For control system engineers: Improve model-based predictions of forced system behavior by fine-tuning time-series models with synthetic data for better sensitivity to input changes.
- •For industrial automation teams: Use enhanced time-series models to simulate what-if scenarios in machinery without retraining, helping diagnose system response under varying input conditions.
Authors
Hong-In Won
Abstract
Covariate-aware time-series foundation models (TSFMs) promise training-free what-if answers for instrumented plants: the change in output that a different future input would cause. We test this on forced engineering systems with exact counterfactuals, comparing Chronos-2, TimesFM-2.5 and TabPFN-TS with classical system identification fitted to the same context. Through their default covariate interfaces, TimesFM-2.5 and TabPFN-TS are memoryless: the predicted effect of an input change is a same-time function of that change ($R^2 = 1.000$ for TimesFM-2.5). Chronos-2 identifies dynamics in context but attenuates them. Its predicted effect is 0.33-0.80 of the true effect, its recovered impulse response has the wrong shape, and its error on a one-degree-of-freedom oscillator levels off at 0.57 with 8192 context samples, where ARX fitted to 256 samples reaches 0.02. Context dither at inference lowers the what-if error on all six synthetic classes without training. A 26-minute fine-tune on synthetic forced systems restores the response magnitude (sensitivity 0.83-0.96) and outperforms structure-agnostic identification on Wiener-Hammerstein and a held-out friction class. A specialised in-context identifier trained on the same data comes close, so the forced-system data carry most of the gain. On three of four measured plants classical identification remains clearly better, and the fine-tuned model loses part of its univariate forecasting skill. Paired counterfactual inputs, together with shuffled future inputs on measured records, test two properties: whether the covariate interface can represent dynamics and whether the pretraining prior covers the plant's time scale. Only the counterfactual pairs expose the attenuation.