Hyperreservoir networks improve prediction of changing time series
Context-dependent time-series prediction via HyperReservoirs
Machine Learning
Summary
Predicting future data points in changing or complex time series is hard because the underlying rules can shift. The authors introduce HyperReservoirs, a special neural network design that uses one part to learn the data and another smaller part to adjust the prediction based on context. This approach keeps training simple but adapts better than older methods. Tests on well-known chaotic systems showed HyperReservoirs predicted future points more accurately than competing models, especially when the data changed speed but not overall behavior.
What this means in practice
- •For data science teams: Enhance forecasting models for systems with shifting behavior by integrating context-modulated reservoirs for better accuracy.
- •For control systems engineers: Improve controllers that predict and respond to changes in dynamic systems by adapting predictions with context-aware reservoir computing.
Authors
Kohei Tsuchiyama, Takatomo Mihana, Ryoichi Horisaki, André Röhm
Abstract
Time series prediction is a common application of reservoir computing. When the training and testing time series data contains multiple dynamical regimes, because an underlying parameter is changing, or the data in fact consists of multiple distinct systems, simple application of the reservoir computing principle produces high prediction errors. Here, we propose a HyperReservoir as an extended model of reservoir computing especially designed for such cases. The HyperReservoir combines a main reservoir with a smaller context reservoir, where the latter modulates the output weights of the former. This structure resembles the hypernetworks from deep neural network literature. However, in contrast, HyperReservoirs retain the simple training via linear regression of standard reservoir computing. We compare the proposed architecture with a conventional ESN, in which context acts at the input, and a full-matrix Conceptor, in which context modulates the reservoir state space. We evaluate all three models on time-series prediction tasks based on Lorenz and Rössler systems, including for varying bifurcation parameters and time sampling scales. We find that the HyperReservoir achieves the lowest mean test error in all three tasks, and particularly outperforms conceptors on data that is sampled from the same attractor but at different time scales.