Which Histories Matter for Time Series Forecasting? Learning Predictive Relevance with Future Supervision
2026-08-24 • Information Retrieval
Information RetrievalMachine Learning
AI summaryⓘ
The authors explore which past time-series data points are truly useful for making future predictions, rather than just assuming similarity means usefulness. They introduce a method that combines a basic similarity-based retrieval step with a machine learning model that ranks candidates based on how relevant they are expected to be for the specific query. Their approach uses information from the future during training to improve ranking but relies only on past data during inference. Tests on various datasets show this method improves prediction relevance in some cases but not universally, indicating that the best retrieval method depends on the domain and data characteristics.
time-series predictionhistorical retrievalpredictive relevancemultilayer perceptron (MLP)retrieval-augmented forecastingcandidate rankinginference-time scoringpattern retrievaldomain dependencequery-specific relevance
Authors
Yong-Hoon Choi, Youngjin Cho
Abstract
Historical retrieval for time-series prediction commonly treats past similarity as a proxy for usefulness. We ask a different question: which historical examples should be expected to matter for a query? We define predictive relevance as expected future utility conditioned on inference-time information, using realized futures only during training as privileged supervision. A normalized-pattern retriever first forms a coarse candidate set, and a lightweight residual multilayer perceptron (MLP) learns a listwise future-compatibility target while keeping inference-time scoring strictly past-only. Our method retains similarity-based candidate generation but reranks its candidates by a more predictive relevance criterion. Optimal relevance decomposes into candidate-level utility and query-specific compatibility, motivating Candidate-Prior and Shuffled-Future controls. Across six benchmarks, the reranker improves Pattern retrieval while revealing candidate-global, query-specific, and mixed relevance regimes. On all 12 confirmatory tasks, it improves Pattern and outperforms a matched-protocol Stationarity-Aware Retrieval-Augmented Time Series Forecasting (SARAF) retrieval rule. Architecture-matched ablations show that correct future supervision, rather than the MLP or added context alone, drives gains in query-specific regimes. Alternative-similarity experiments show that a strong last-value-anchored L2 rule remains superior in some domains, whereas future-supervised relevance is particularly strong where our diagnostics indicate query-specific relevance, especially on Solar. Candidate-pool diagnostics show that this contrast is not explained solely by coarse Pattern retrieval. Overall, historical relevance is structured and domain dependent rather than governed by a universally superior retrieval rule.