PhysForge enables detailed 3D object reconstruction with material control
PhysForge: Material-Controllable 3D Reconstruction with Separable Components
Computer Vision and Pattern Recognition
Summary
Reconstructing 3D objects that behave realistically when touched or moved is hard, especially when you want to control their materials like rubber or metal. The authors developed PhysForge, a system that uses videos and smart modeling to figure out the physical properties of object parts and how they interact. This lets creators simulate objects with different materials more accurately and mix those parts realistically. PhysForge was tested on real scanned objects and showed better control and realism compared to earlier methods.
What this means in practice
- •For 3d graphics developers: Create 3D models with precise physical material properties for realistic animations and simulations involving complex object interactions.
- •For robotics engineers: Use detailed, physically accurate 3D object models to improve robot manipulation and interaction with various materials in simulated environments.
Authors
Yuchen Wang, Yanqi Bao, Zheng Gu, Zhengqing Chen, Jieqi Shi, Jing Huo
Abstract
Recent advances in physical 3D generation have enabled visually faithful assets with simulatable mechanical behavior. However, existing methods provide limited control over material behavior beyond appearance-based inference and struggle to estimate physical properties from complex interactions between objects and semantic parts. Therefore, we present PhysForge, a Video Diffusion Model (VDM)-based framework for addressing these challenges, comprising two complementary modules for material-controllable physical fitting and componentlevel interaction modeling. The first, PhysFit, leverages material-conditioned VDM dynamics as references for physical-property fitting, introducing windowed fitting to focus on material-revealing interactions and Hallucinated Forcing to disentangle nonphysical motion hallucinated by the VDM from material responses. The second, PhysComp decomposes complex interactions into individual objects and semantic components, enabling their physical parameters to be optimized independently and flexibly composed for joint simulation. Experiments on PhysAssets and Google Scanned Objects demonstrate that, compared with existing methods, PhysForge provides stronger control over material-dependent physical responses while supporting component-level physical modeling for complex interactions. It achieves the best CLIP and physical controllability scores on both common and uncommon materials, with qualitative comparisons, component-level ablations, and a user study further supporting these improvements. Our project page is available at https://masterwanna.github.io/PhysForge.html .