Stationary infinite memory chains yield adaptive empirical confidence bounds
Empirical concentration for chains with infinite memory
Machine Learning
Summary
Predicting time series data is hard because the data points depend on a long history, which is often unknown and complicated. The authors develop a new way to measure how much past data influences the present, even when the history goes infinitely far back. They then show how to estimate this influence from just one observed data sequence, making predictions more confident and reliable. Their method adapts automatically without needing to know hidden constants, improving how we handle complex time-dependent data.
What this means in practice
- •For time series analysts: Generate data-driven confidence intervals for predictions in complex dependent time series without prior parameter knowledge.
- •For statistical modelers: Estimate dependence structure from observed sequences to improve model reliability for processes with long memory effects.
Authors
Pierre Alquier, Paul Doukhan
Abstract
Concentration inequalities for time series usually involve unknown quantities measuring dependence. This prevents their direct use as confidence bounds, or as criteria for learning algorithms. Recent works proved empirical concentration inequalities for stationary Markov chains, where the unknown quantities are estimated. We extend this approach beyond the Markov case, by considering stationary chains with infinite memory. We first establish a bounded-differences inequality on bounded Polish metric spaces under a summable Wasserstein contraction condition. We then estimate its dependence coefficients from a single trajectory, using either transition counts when the state space is finite, or derivatives of kernel conditional-density estimates for a continuous state space. We give general error bounds and illustrate their dependence on the memory tail. Finally, a fixed multiplicative safety factor turns these estimators into empirical concentration bounds. While some regularity assumptions are required to make the bounds empirical, our procedure remains fully adaptive in the sense that it does not require the knowledge of the constants involved in these assumptions. The empirical confidence bounds derived from our results are asymptotically as tight as the theoretical ones.