Streaming recurrent memory improves learning from continuous data streams
Streaming Recurrent Memory
Machine LearningArtificial Intelligence
Summary
Most AI systems learn by storing and reusing past experiences, which can be bulky and slow. The authors created Streaming Recurrent Memory (SRM), a kind of memory that learns from data as it arrives without storing everything. SRM remembers important information and adapts even when feedback is delayed, all while using an efficient one-step learning process. In tests, SRM did better than other memory methods at recalling information and handling tasks where rewards come much later.
What this means in practice
- •For real-time system developers: Build AI that learns and adapts instantly from ongoing sensor data without storing raw past inputs.
- •For robotics engineers: Enable robots to remember and act on experiences received continuously, including with delayed rewards in complex environments.
Authors
Mohamed Elsayed, Minh Pham, A. Rupam Mahmood
Abstract
Natural intelligence learns from a continuous stream of sensorimotor experience: data can arrive at high bandwidth, and behavior adapts to feedback as soon as it arrives, even when it is delayed. In contrast, most modern deep learning systems rely on stored datasets, replay buffers, and retained contexts through which credit is later assigned. At high data rates, storing and revisiting past experience can be impractical under tight memory and computational constraints. This paper studies recurrent memory under a streaming learning constraint: the learner learns from experience as soon as it arrives without storing raw experience. We ask whether a streaming learner can maintain a state that remains useful for future prediction and control from delayed feedback. We introduce Streaming Recurrent Memory (SRM), a recurrent memory designed for the streaming setting which preserves information that may matter later while remaining trainable with one-step backpropagation through time (BPTT). SRM provides a streaming-compatible recurrence that is designed to recall past information by content or time, track elapsed steps, and compose transformations whose outcome depends on the order in which inputs are observed. Across controlled memory tasks and streaming applications, SRM learns effectively with one-step BPTT, outperforming recurrent baselines in associative recall and online language modeling while achieving near-perfect success in associative T-Maze with delayed rewards. These results suggest that fixing how inputs are stored in the recurrent state, while learning how that state is read out, enables effective streaming learning without long-range backpropagation.