Neural networks change system behavior and need ongoing adjustment
Learning Changes the World and Then What Was Learned has Changed: Training and Deploying Neural Networks in the Control Loop
Machine Learning
Summary
Neural networks, when used to control systems, can change how those systems behave in ways that differ from their original training. The authors identify specific problems that happen when these networks are applied without updating, causing control errors to grow over time. They analyze why these errors occur and propose a way to continuously adjust the networks during operation. Tests on a simulated complex system show that ongoing learning can keep control accurate, unlike fixed networks.
What this means in practice
- •For control system engineers: Maintain accuracy in feedback systems by using continuous neural network updates to adapt to changing system behaviors.
- •For robotics software developers: Implement online learning in robot controllers to reduce errors caused by shifts in operational conditions after deployment.
Tested on simulated data.
Authors
Vittorio Lippi
Abstract
Neural networks offer critical adaptability for controlling complex systems where nominal modeling proves intractable. However, when offline-trained neural networks are deployed within closed-loop control systems, their actions fundamentally alter the state distribution upon which they were trained. This paper introduces a unified taxonomy to categorize these sequential failure modes: the initial Closed-Loop Identification Error, the deployment-induced Action Error, and the continuous Tracking Error. Through discrete-time sensitivity analysis, this work formalizes how offline-learned parameters locally guarantee a trajectory divergence upon deployment, degrading performance due to local loss geometry. To mitigate this distribution shift, we propose actionable design policies centered on time-scale separated online adaptation. These mechanisms are validated through ablation studies on a nonlinear Duffing oscillator, demonstrating that bounded continuous learning successfully mitigates trajectory divergence whereas static deployments fail, offering concrete guidelines for practitioners deploying machine learning components in safety-critical feedback systems.