Solver free training for differential equations on microcontrollers succeeds
TinyUDE: Solver-Free Universal Differential Equations on Microcontrollers via Lie-Taylor Jet Matching
Machine LearningEmerging Technologies
Summary
Training models that describe systems using differential equations usually needs complex computations that use too much memory for tiny devices like microcontrollers. The authors developed a new method called Lie-Taylor jet matching that trains such models without relying on these heavy computations, saving memory. They tested it on devices like an ESP32 microcontroller and found it works just as well as traditional methods while using far less memory and running in real-time. This makes it possible to train dynamic system models directly on small devices without extra computing power.
What this means in practice
- •For embedded system developers: Run and train physics-based models on microcontrollers with limited memory and compute resources using solver-free methods.
- •For robotics engineers: Implement accurate, on-device system identification for dynamic robots without relying on heavy numerical solvers.
Authors
Pranavanath Balamurali, Hrishi Kamireddy
Abstract
Training Universal Differential Equations (UDEs) traditionally relies on backpropagating through numerical ODE solvers, creating memory footprints far exceeding the capabilities of edge microcontrollers. We present Lie-Taylor jet matching, a solver-free training framework that fits a hybrid vector field directly to the first and second time-derivatives of observed system states. These derivatives, the truncated Lie-Taylor jet, are estimated online via Savitzky-Golay filtering, yielding fully analytic gradients without automatic differentiation software. We evaluate whether eliminating the solver compromises accuracy against a conventional baseline (fixed-step RK4 integration, multiple shooting, exact discrete adjoints, Adam) sharing identical dynamics, noise models, network architectures, and metrics. While naive derivative matching degrades under sensor noise, our noise-adaptive mechanisms close and reverse this gap: full-rate phase-shifted sampling, a reservoir buffer, cosine-annealed optimization with weight averaging, on-device noise estimation, and polynomial-misfit quality gating. On a damped pendulum and chaotic double pendulum, our method matches or exceeds baseline accuracy at matched data windows and recovers unmodeled damping coefficients. Across noise levels from 0% to 5%, it attains a geometric-mean relative field error of 0.65x that of the baseline within 108 kB of static memory, compared with megabytes of solver tape. On an ESP32 microcontroller, the on-device run reaches a field error of 0.0020 and recovers the damping coefficient to c = 0.400 (true 0.400) within 61.3 kB of static memory and 7.24 ms per update (18.1% duty cycle at 25 Hz), confirming real-time on-device training is feasible without a numerical solver.