Coverage aware prompt selection improves time series forecasting with large language models
Beyond Similarity: Coverage-Aware Prompt Selection for Time Series Forecasting with LLMs
Machine LearningComputation and Language
Summary
When large language models predict future events based on past patterns, they usually pick examples that look very similar to recent data. This method tends to ignore important rare events that could help improve predictions. The authors introduced a way to select prompts that track how often different types of examples are used, making the model more aware of covering diverse patterns. Their approach improved forecasting accuracy on several datasets, showing benefits over standard similarity-based methods in most cases.
What this means in practice
- •For data scientists: Improve time series predictions by selecting diverse prompts that better represent rare but important temporal patterns.
- •For forecasting teams: Enhance demand and resource forecasting accuracy by integrating coverage-aware prompt selection in large language model workflows.
Authors
Daeun Ji, Minkyoung Kim, Dongkuk Kim, Yohan Lee, Beomsoo Kim, Beakcheol Jang
Abstract
Similarity-based retrieval is the dominant rule for conditioning large language models (LLMs) in in-context learning, retrieval-augmented generation, and prompt-based time series forecasting. The rule concentrates on near-duplicate candidates, an issue that has motivated diversity-aware retrieval but remains unexamined in other retrieval-conditioned pipelines. We study this issue using prompt-based time series forecasting as a test bed, where a learned prompt pool is retrieved by similarity. Dominant methods in this setting retrieve top-K entries by cosine similarity without redundancy control, producing a bias toward dominant temporal patterns while overlooking rare but informative events. We propose CASP-LLM, a coverage-aware semantic prompting framework that addresses this prompt selection bias by combining usage-tracking and saturating-gate techniques into a coverage regularizer that adds no learnable parameters. On six long-term benchmarks and the M4 short-term benchmark, CASP-LLM matches or improves on similarity-based LLM forecasters on most dataset-horizon settings, with the exceptions of Electricity, M4-Monthly, and the few-shot long-horizon setting. A controlled study locates the failure mode at the cross-batch usage level rather than per-retrieval redundancy: within-retrieval diversification such as MMR does not help, whereas regularizing anchor usage across training does.