Learnable lens networks predict long-term dynamics with fewer parameters
LLN: Learnable Lens Networks for Parameter-Efficient Long-Horizon Dynamical Prediction
Machine Learning
Summary
Long-term prediction of how things change over time is hard for computers because errors build up. The authors introduce a new type of neural network called Learnable Lens Networks (LLN) that uses ideas from physics to better track these changes. Instead of simply adding updates directly, LLN simulates how light moves and bends, making predictions more stable and accurate over many steps. Their experiments show LLN can predict future states longer while using fewer resources.
What this means in practice
- •For engineering simulation teams: Enable more efficient and stable long-term simulations of physical systems with fewer neural network parameters.
- •For robotics control developers: Improve prediction models for robot dynamics to enhance control over extended time horizons with less computational cost.$Commercial implications: This paper enables building lightweight, accurate models that can be integrated into commercial robotic control systems for better long-term planning.
Authors
Binbin Yong, Zhao Su, Lan Guo, Haoran Li, Jun Shen, Qingguo Zhou
Abstract
Explicit residual connections of the form (x+f(x)), often combined with normalization layers, have become a standard strategy for training very deep neural networks. However, residual addition primarily provides an algebraic shortcut for gradient propagation, while leaving the evolution of feature geometry across layers largely unconstrained. We introduce Learnable Lens Networks (LLN), a physics-inspired architecture that replaces direct feature-space residual accumulation with learnable optical transport in an augmented position-angle phase space. Each layer alternates between free propagation, which provides an implicit transport path, and a learnable lens field that performs nonlinear trajectory transformation and focusing. Theoretically, we establish that LLN transport is globally invertible and volume-preserving for any differentiable lens field, with the implemented coordinate-wise Gaussian transport further satisfying symplecticity. Importantly, these structural constraints do not limit expressivity: with unrestricted embeddings and readouts, LLN retain universal approximation of continuous end-to-end maps. Experiments across diverse dynamical systems demonstrate that LLN improves long-horizon prediction while using substantially fewer parameters than same-depth comparators. Further analysis reveals stable depth-wise gradient transport and interpretable learned dynamics under the coupled propagation and refraction design.