Analog oscillatory hardware learns with lock-in equilibrium propagation
Lock-in EP: An In-Situ Training Algorithm for Oscillatory Hardware
Machine Learning
Summary
Analog computers could use less energy than digital ones, but their parts aren’t always reliable. The authors developed a training method called lock-in equilibrium propagation (LIEP) that helps analog systems learn by updating themselves without extra steps. This method can train networks from scratch or fix them after their settings get changed. Their approach may help analog devices learn directly on the hardware, although it has so far only been tested on simple networks.
What this means in practice
- •For analog hardware engineers: Enable analog oscillatory devices to train themselves directly on the hardware without separate forward and backward passes.
- •For embedded systems developers: Recover or adjust pre-trained analog network performance after component disturbances without rebuilding the entire system.
Authors
Sowjanya Tammali, Wilkie Olin-Ammentorp
Abstract
Analog hardware platforms offer the potential to reduce energy consumption over digital architectures, but in order to succeed, large-scale analog systems must also be able to operate with or recover from the variability of their components. Towards this goal, we derive and demonstrate the lock-in equilibrium propagation (LIEP) training method. LIEP provides local gradient information for each component in an oscillatory network without separate forward and backward sweeps, potentially allowing for in-situ learning capabilities on analog oscillatory hardware platforms. We demonstrate that LIEP can be used both for ab-initio training as well as recovering performance when pre-trained parameters are perturbed. We show that LIEP can be formulated as a three-factor update rule, and suggest that although the method is currently only validated on shallow networks, alternate architectures may allow it to extend to deep and large-scale networks addressing complex tasks.