When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters
2026-07-06 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied when complex time series models called foundation models (FMs) are worth using over simpler methods like XGBoost. They found that FMs do better on half of the tested datasets regardless of how much data is used, justifying the cost of GPU deployment for those cases. For some datasets, simpler methods work well even with very small training sets, while for others, you need a moderate amount of data to see FM benefits. They also showed that a quick rule based on dataset size and seasonality can decide when to use zero-shot FMs without training, saving effort. Their work provides a practical guide for choosing between foundation and classical models before heavy investments.
time seriesfoundation modelXGBoostzero-shot learningLoRA fine-tuningseasonalitybenchmark datasetstraining sizeGPU deploymentbreak-even analysis
Authors
Nicholas Tan Jerome, Frank Simon
Abstract
Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We provide the first systematic break-even analysis answering when this investment pays off. Across 30 benchmark datasets, we compare zero-shot and LoRA fine-tuned foundation models (Chronos, Moirai, Lag-Llama) against classical baselines (Naive, ETS, ARIMA, XGBoost) at six training set sizes from 2% to 100% of available data. Foundation models outperform classical methods at every evaluated training fraction on 15 of 30 datasets - GPU deployment is unconditionally justified on these regardless of data volume. On 6 datasets, classical methods surpass zero-shot foundation models with as little as 2% of training data (21-2,768 samples); on the remaining 9, break-even ranges from 24 to 8,361 samples. One robust deployment rule requires no model training: if n_train < 700 and seasonality is non-negligible, use FM zero-shot and skip fine-tuning - this resolves 10 of 30 deployment decisions immediately. Contrary to common practice, LoRA fine-tuning can actively degrade performance on short series. We operationalise these findings as a two-step decision framework - compute dataset length and seasonality strength, run a brief 5-10% pilot only if needed - enabling practitioners to make the FM-versus-classical decision before committing to full infrastructure. Four dataset features motivate mechanistic hypotheses for the remaining cases, though reliable automated prediction at this benchmark scale remains an open problem. Code, benchmark, and decision tools are available at https://github.com/nicolaisi/fm-breakeven.