Atemporal model finds exact timing when one signal affects another
ATEMPO: Amortized Transfer Entropy Model Per-lag Over-Time
Artificial IntelligenceInformation Theory
Summary
Sometimes one sequence of data influences another, but only after a certain delay. Existing methods struggle to identify exactly when this influence happens, mixing helpful and unhelpful information. The authors created ATEMPO, a model that can learn these delays quickly and accurately across different times and lags without needing to be retrained multiple times. This helps find the right timing of influence and improves predictions in weather forecasting and brain-computer interfaces.
What this means in practice
- •For weather forecasters: Identify exact delays in meteorological data to improve forecasting accuracy with fewer input features.
- •For brain-computer interface developers: Select optimal time lags in neural signals to improve performance of brain-controlled devices.
Authors
Emma Andrews, Konstantinos Kontras, Keane Ong, Maarten De Vos, Gianmarco Mengaldo
Abstract
When one time series informs another, the delay at which that information arrives determines if models can utilize this relationship effectively. A source may be strongly predictive at one delay but uninformative at neighboring lags. Therefore, fixed input windows may obscure this structure by bundling informative and irrelevant history together. Transfer entropy (TE) can recover these delays by measuring how much a source's past tells us about a target beyond the target's own history. However, existing TE estimators require separate fits across source and conditioning depths, with further estimation on local windows to track delays over time. We introduce ATEMPO, a diffusion-based transfer entropy estimator amortized over lags and time. ATEMPO samples source and conditioning depths during training and conditions explicitly on temporal position, allowing a single score network to estimate the full TE surface without refitting. Differencing this surface recovers when information arrives. Across synthetic datasets with ground-truth TE, ATEMPO achieves the lowest mean relative error among baselines, and on datasets with known lags, ATEMPO achieves $0.98\pm0.04$ lag-detection accuracy. ATEMPO requires a single fit across source depths $K=6$ and target depths $L=3$, whereas existing estimators require 18 separate fits to evaluate the same $K \times L$ grid. On three systems with time-varying lag, ATEMPO is the only estimator to maintain perfect lag-detection accuracy across all evaluated window lengths. These recovered delays improve downstream forecasting: ATEMPO-selected inputs reduce MSE on WEATHER-5K by 7.25%, while 120 selected inputs outperform the benchmark's 240-feature default by 4.6%. Across three brain-computer interface tasks, using ATEMPO-selected lags achieves the highest or equal-best $R^2$, improving mean $R^2$ by 4.1% relative to the strongest competing baseline.