Sector-ring linear reservoirs cut search cost while improving prediction accuracy
Sector-Ring Linear Reservoirs: Sector-Wise Decay-Timescale Allocation With State Reuse Across Candidate Settings
Machine LearningNeural and Evolutionary Computing
Summary
Predicting complex, chaotic systems is hard because each task needs a different memory length and response speed from a fixed computer model called a reservoir. The authors designed a special reservoir that organizes its internal behavior into groups with different decay rates, which can be reused to speed up searching for the best settings. Their design reduces computation costs drastically while matching or beating other state-of-the-art methods on chaotic prediction tasks. The way the memory timescales were assigned changed depending on what was being predicted and for how long.
What this means in practice
- •For time series modelers: Speed up tuning of models for predicting chaotic or complex time sequences by reusing computations across parameter settings.
- •For signal processing engineers: Design systems with adjustable memory timescales tailored to different frequency components for improved forecasting accuracy.
Authors
Yudai Ebato, Sou Nobukawa, Yusuke Sakemi, Takashi Kanamaru, Nina Sviridova, Kazuyuki Aihara
Abstract
In reservoir computing, performance depends on whether the fixed dynamics of the reservoir suit the task, and the appropriate memory length and frequency response differ across tasks. The reservoir settings are found by a search on each task, and driving the reservoir with the input and collecting its states for every candidate setting dominates the cost of the search. In a reservoir with a linear recurrence, the dynamics decompose into modes, one per eigenvalue, and the eigenvalue distribution becomes a design variable. We propose the Sector-Ring Linear Reservoir (SRLR), in which the eigenvalue angles are fixed at evenly spaced positions around a circle and one decay timescale, chosen from a few values, is assigned to each sector of consecutive angles. To keep the scale of each mode's state equal across candidates, the input weights are scaled using the power spectrum of the training input. To reduce the cost of the search, the states of the modes for each decay-timescale value are computed once and reused across candidates. On prediction and reconstruction benchmarks built from chaotic systems, the SRLR achieved, in all eight cases, a lower mean test error than a reference model in which all modes share one decay timescale, and was competitive with nonlinear echo state network and leaky-integrator echo state network baselines. The selected decay-timescale profiles changed with the prediction target and horizon. The state reuse reduced the number of computed mode trajectories in the search by 99.8% relative to per-candidate recomputation.