CoDimRecon reconstructs editable 3D scenes with deformable and rigid objects
CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes
Computer Vision and Pattern RecognitionGraphicsRobotics
Summary
Building realistic 3D scenes from photos usually works best with solid, unchanging objects. The authors created CoDimRecon, a method that can rebuild scenes including flexible things like ropes, shells, and soft solids by representing them as curves, surfaces, or volumes. Their system also figures out how parts are connected and can test if the models behave like real objects, fixing issues along the way. This helps to create 3D models that can be directly used in simulations for robots and virtual environments.
What this means in practice
- •For robotics developers: Create detailed, editable 3D models from photos that include flexible and jointed objects for realistic robot simulation and interaction testing.
- •For game environment designers: Generate 3D scenes with deformable elements that respond realistically in game physics engines using multi-view image inputs.
Authors
Shuzhao Xie, Lelin Wang, Guying Lin, Zhi Wang, Minchen Li
Abstract
Reconstructing simulation-ready 3D scenes from real-world observations enables robotics, gaming, and immersive applications, yet existing methods largely assume rigid objects. This leaves an important gap for deformables, whose simulation-ready geometry depends on dimensionality (curves, surfaces, or volumes) and whose behavior may require models beyond elasticity. We present CoDimRecon, an agentic framework that reconstructs editable scenes containing rigid, articulated, and deformable objects from multi-view RGB observations. Scene-level geometric priors ground scale and layout, while object-level generated meshes guide the agent toward detailed, compact geometry; articulated rigid objects are decomposed into movable parts with explicit joints. For deformables, category-wise agent sessions reconstruct curves as centerlines with radii, surfaces as manifold shells with thickness, and volumes as watertight solids for volumetric meshing. Reusable simulator skills initialize compatible physical models and parameters, while agent-guided behavioral tests expose mismatches and trigger targeted revisions of motion, geometry, numerics, or material modeling. On evaluated Replica and ScanNet++ scenes, CoDimRecon achieves competitive compositional reconstruction accuracy while additionally producing deformable assets for rod, shell, and solid simulation. We further demonstrate robot interactions across all three representations, including a controlled paper-folding case in which behavioral testing motivates plastic bending.