PhysReal models real-world squishy objects from videos for better predictions
PhysReal: Learning Real-World Deformable Object Physics via Hybrid Constitutive Modeling
Robotics
Summary
It is hard to understand how soft, squishy things move and behave just by watching them because they are made of different materials in different spots. The authors created PhysReal, a computer program that learns how these objects work by using videos and a mix of simple physical models with smart neural networks. This program breaks down an object into many small areas, each with its own material rules, and improves the rules step by step to fit what it sees in the videos. PhysReal can then accurately guess how the object will move or change in the future, which could help robots interact with real-world soft items more effectively.
Deformable objectsConstitutive modelMaterial heterogeneityNeural networksDifferentiable simulationMaterial point method (MPM)3D renderingPhysics learningRoboticsDynamic reconstruction
Authors
Yinan Deng, Jianqiao Song, Yisi Zhang, Yuhan Wang, Jiahui Wang, Yufeng Yue
Abstract
Learning physically plausible dynamics from visual observations is essential for interactive world models and embodied agents. However, modeling real-world deformable objects remains challenging because their dynamics often arise from complex, spatially heterogeneous material responses. To address this challenge, we propose PhysReal, a video-driven framework for learning and simulating the underlying physics of real deformable objects. PhysReal integrates a spatially varying hybrid expert-neural constitutive model with a differentiable MPM simulator and 3DGS renderer. Analytical expert models provide interpretable physical priors, while neural constitutive residuals capture material responses beyond predefined formulations. Spatially distributed patches parameterize the constitutive field, enabling a continuous representation of local material variations. To organize the identification of this model from sparse visual observations, we adopt a progressive curriculum that sequentially optimizes global material properties, spatially varying local parameters, and neural constitutive residuals, together with complementary motion and mask supervision. Extensive experiments on diverse deformable-object interactions demonstrate that PhysReal achieves superior performance in dynamic reconstruction and future-state prediction, while showing strong potential for downstream robotic applications.