PhysReal learns real-world deformable object physics from videos
PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling
Robotics
Summary
It's hard to understand how soft or squishy objects like cloth or rubber move in the real world because they bend and stretch in complicated ways. The authors created PhysReal, a system that watches videos of these objects moving and learns how their materials behave by combining known physics rules with smart neural networks. This system can then predict how these objects will move in the future and can help robots interact with them more naturally. PhysReal carefully figures out global and local material properties step-by-step to better model different parts of an object.
What this means in practice
- •For robotics engineers: Enable robots to better predict and manipulate soft deformable objects by providing accurate learned physics models from video data.
- •For computer graphics developers: Create more realistic animations of deformable materials by using learned hybrid physics models that reflect real-world behavior.
Authors
Yinan Deng, Jianqiao Song, Yisi Zhang, Yuhan Wang, Jiahui Wang, Yufeng Yue
Abstract
Learning physically plausible dynamics from visual observations is essential for interactive world models and embodied agents. However, modeling real-world deformable objects remains challenging because their dynamics often arise from complex, spatially heterogeneous material responses. To address this challenge, we propose PhysReal, a video-driven framework for learning and simulating the underlying physics of real deformable objects. PhysReal integrates a spatially varying hybrid expert-neural constitutive model with a differentiable MPM simulator and 3DGS renderer. Analytical expert models provide interpretable physical priors, while neural constitutive residuals capture material responses beyond predefined formulations. Spatially distributed patches parameterize the constitutive field, enabling a continuous representation of local material variations. To organize the identification of this model from sparse visual observations, we adopt a progressive curriculum that sequentially optimizes global material properties, spatially varying local parameters, and neural constitutive residuals, together with complementary motion and mask supervision. Extensive experiments on diverse deformable-object interactions demonstrate that PhysReal achieves superior performance in dynamic reconstruction and future-state prediction, while showing strong potential for downstream robotic applications.