LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications
2026-08-24 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors review how large language models (LLMs) are used to make predictions about future events by combining language understanding with data and tools. They categorize different system designs, including ones that just use LLMs, those that include extra information or tools, and combinations with other types of models. They discuss how these systems are trained and tested, noting that sometimes the LLMs do not actually improve accuracy or results may be misleading due to data issues. The authors also highlight practical uses in areas like finance and weather, and emphasize challenges like measuring performance and dealing with changing conditions.
large language modelsforecasting systemstime seriesevidence retrievalexternal toolshybrid systemstraining methodsevaluation protocolsdistribution shiftbenchmark contamination
Authors
Xiaogang Xu, Jiaqi Tang, Jianmin Chen, Yingying Yan, Zhenchao Tang, Xiangxin Zhou, Xiaobin Hu, Wei Wei, Jinfeng Wu, Qifeng Chen, Lu Zhou, Jiafei Wu, Zhe Liu, Jianwei Yin, Weimin Zheng
Abstract
Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external tools, and iterative prediction. We investigate LLM-based forecasting agents, meaning systems in which a language model contributes to a scored prediction about a future or currently unobserved target. We organize architectures into three groups. Standalone LLM workflows operate on encoded time series or event context. Tool- and retrieval-augmented agents incorporate external evidence. Hybrid systems pair LLMs with statistical or foundation models. We then review training methods and evaluation protocols. We examine negative as well as positive evidence, including sensitivity to small input perturbations, ablations in which the LLM component does not improve accuracy, and benchmark gains that may reflect contamination instead of temporal reasoning. We cover applications in finance, weather, health, energy, and operations, and we summarize the benchmarks and datasets used for evaluation. The evidence indicates that measurement is a central limitation. Future work requires calibration under distribution shift, contamination-resistant live evaluation, explicit reporting of cost and accuracy together, and methods for handling feedback between deployed forecasts and the outcomes being forecast.