Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos
2026-08-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the challenge of understanding how objects move just by looking at videos, which is hard because pictures don’t directly show physical properties. They created a new system called Analytic Dynamics that uses an extra step with physics-based information, like object positions and deformations, to help computers better learn object motion. This method trains models using simulated data that include both video and exact physical details, making the learning more accurate and reliable. Their experiments show that this approach can quickly and correctly figure out how objects behave from regular videos.
object dynamicsvisual observationsphysical simulationdeformation gradientfeed-forward modelparameter regressionmaterial model classificationinductive biasmonocular videophysical state trajectories
Authors
Jailing Lin, Jikuan Zhang, Jianhua Sun
Abstract
Inferring object dynamics from visual observations is essential for intelligent agents to reason about and interact with the physical world, yet remains challenging due to the fundamental gap between visual evidence and intrinsic dynamics. Existing methods either rely on costly per-scene optimization, limiting efficiency and scalability, or directly map visual evidence to intrinsic dynamics without intermediate physical abstractions, making them prone to appearance and geometry shortcuts. To bridge this gap, we propose Analytic Dynamics, a feed-forward dynamics inference framework that introduces an intermediate physics-grounded dynamics representation between visual observations and intrinsic dynamics. Specifically, we leverage privileged physical states, including position, displacement, and deformation gradient fields, which are available in simulation, to learn a structured dynamics representation that is difficult to discover from visual observations alone. By aligning visual representations with this space, we equip visual models with a physics-grounded inductive bias, guiding them to capture dynamics-relevant patterns for material model classification and parameter regression. To facilitate this research, we develop a dynamics data generation pipeline and benchmark containing paired physical state trajectories, rendered videos, and ground-truth material models and parameters. Extensive experiments demonstrate that Analytic Dynamics achieves efficient, accurate, and generalizable dynamics inference from monocular videos.