Rgbd depth maps refine 3d object meshes faster with symmetry
RGBD-to-3D Object Mesh Refinement via Depth Matching and Symmetry Propagation
Computer Vision and Pattern Recognition
Summary
3D models made from single pictures often don't match perfectly with the original view, especially around edges and hidden parts. The authors introduce a fast way to fix these 3D models using depth information from RGBD images, without needing to retrain existing systems. Their method uses depth matching to correct visible areas and copies these fixes to hidden areas based on object symmetry, improving accuracy quickly. This approach works well with different existing methods and can handle noisy real-world data.
What this means in practice
- •For 3d graphics developers: Enhance existing 3D object reconstruction tools by incorporating fast depth-based mesh refinement improving visible and hidden surfaces without retraining.
- •For robotics engineers: Improve 3D object models from robot-mounted RGBD cameras to better handle occlusions and sensor noise, aiding perception and manipulation tasks.
Authors
Ahyun Seo, Minsu Cho
Abstract
Single-view 3D reconstructors often produce plausible meshes that disagree with the input view, especially near depth discontinuities and self-occlusions. We present a lightweight, plug-and-play RGBD-to-3D refinement that improves any RGB-to-3D reconstructor without retraining. Given a depth map, we correct the visible surface by bipartite matching to back-projected depth points, mirror these corrections onto the occluded side across a detected symmetry plane, and propagate them with a smoothness solver. Every stage is closed-form, making the method orders of magnitude faster than optimization-heavy test-time refinement. On GSO and OmniObject3D with five backbones, it yields consistent gains, also with monocular pseudo-depth, benefits more from symmetry on symmetric objects, and compares favorably with prior refinement in accuracy and runtime. It further improves an RGB-D-to-mesh reconstructor and transfers to real captures with noisy sensor depth.