Analog neural networks learn locally with noisy memory devices
Local learning with imperfect analog synapses
Emerging Technologies
Summary
Neural networks running on special analog memory chips often struggle to learn because their updates can be noisy and the usual learning methods need complicated processes. The authors created a new local learning rule called Channel-wise Spatial Gradient (CwSG) that trains each layer by looking only at local features without complicated backward passes. Their simulations showed CwSG works well with real-world noisy analog memories, needing far fewer updates than traditional methods and even improving on very variable devices. This approach shifts the challenge of learning from fixing hardware to smart algorithm design.
What this means in practice
- •For edge device developers: Implement low-power neural network training on analog memory chips that adapt locally without complex hardware or high-precision updates.$Commercial implications: Enables production of edge AI devices that can learn locally with inexpensive, noisy analog memory hardware, reducing energy and complexity.
- •For embedded systems engineers: Design adaptive signal processing systems that update model parameters directly on analog hardware with minimal error correction.
Authors
Théo Dupuis, Adrien Renaudineau, Guillaume Bourgeois, Joel Minguet Lopez, Gabriele Navarro, Jean-Michel Portal, Elisa Vianello, Jacques-Olivier Klein, Damien Querlioz
Abstract
Analog in-memory computing delivers efficient neural network inference at the edge, but systems deployed in non-stationary environments may need to adapt in situ to locally acquired data. Backpropagation faces two structural obstacles here. Its global error transport lets stochastic write errors compound across layers, and its backward pass demands bidirectional, high-precision peripheral circuitry absent from inference-oriented crossbar arrays. We introduce Channel-wise Spatial Gradient learning (CwSG), a strictly local, forward-only rule that trains each convolutional layer by rewarding spatial variation in its feature maps. In simulations replaying the measured trajectories of 1,189 phase-change memory and 1,268 filamentary memristor devices, CwSG learns with single, unverified, sub-1 V pulses. It matches backpropagation on phase-change memory with 59-fold fewer pulses and outperforms it by 53 percentage points on highly variable filamentary memristors, with 8-bit forward converters and without transposed crossbar reads or network-wide activation storage. Analog training thus becomes a problem of credit assignment, in which algorithmic design rather than hardware compensation alone overcomes severe programming noise.