Spiking neural networks improve time-series forecasting with feedback loops
Model- and Layer-Level Closed-Loop Forecasting with SSM-Based Spiking Neural Networks
Machine LearningArtificial Intelligence
Summary
Forecasting future data points from past information is challenging, especially when using spiking neural networks (SNNs), which communicate with binary spikes rather than normal values. The authors explore two ways to let these networks use their own past predictions to forecast further ahead, called model-level and layer-level closed-loop forecasting. They tested their ideas on a standard time-series benchmark and found that their methods match or beat other leading approaches. This shows that SNNs can be competitive for forecasting even when using discrete communication.
What this means in practice
- •For time-series analysts: Use closed-loop spiking neural networks to forecast complex time-series with discrete spike-based computations.
- •For embedded system engineers: Implement spiking neural networks for efficient low-power forecasting in devices where continuous activations are costly.
Authors
Erik Stolt, Sanja Karilanova, Ayca Ozcelikkale
Abstract
Deep State-Space Models (Deep SSMs) have shown promising performance and computational benefits in processing of time-series data with complex temporal dependencies. Spiking neural networks (SNNs) constitute a biologically inspired class of deep SSMs, where computing blocks communicate through spikes, discrete single-bit events rather than continuous activation functions. A particularly important task in time-series processing is forecasting, i.e. predicting future values of a sequence from past observations alone. However, how to best perform forecasting with SNNs is an open question, where the standard autoregressive nature of the data processing with SNNs requires values that are not available during the forecasting phase. In this article, we address this challenge. We explore two closed-loop forecasting strategies, in which the network's own generated values are fed back into the network as if they were observed past values: (i) model-level and (ii) layer-level closed-loop forecasting. We evaluate these approaches on the ETT-small datasets, a standard benchmark for time-series forecasting, under both spike-based and continuous activation functions. We compare the proposed closed-loop approaches against state-of-the-art methods such as SpaceTime, as well as the zero-masking baseline in S4. Both of the proposed approaches perform comparably to or better than zero-masking with S4, and in most cases match SpaceTime. These findings highlight the trade-off between continuous and low-bit quantized activations, and demonstrate that the proposed Deep SSM architectures provide a competitive framework for forecasting.