Harmony creates detailed 3D indoor scenes from single photos
HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis
Computer Vision and Pattern Recognition
Summary
It is hard to build accurate 3D models of indoor spaces from just one picture because existing methods either get the object layout wrong or miss fine details. The authors developed Harmony, a system that first figures out the room layout and orders objects in a smart way, then places them step-by-step using a combination of reasoning and geometric checks. This helps the 3D scene match the photo visually while keeping realistic object relationships. Their tests show Harmony builds better 3D scenes than previous methods from single images.
What this means in practice
- •For augmented reality developers: Generate detailed and spatially correct 3D room layouts from single photos for AR content placement.$Commercial implications: Enables AR companies to create accurate indoor environments from user photos improving virtual object integration.
- •For interior design software teams: Automatically reconstruct room layouts and furniture arrangements from single images to speed up design proposals.
Authors
Shufan Sun, Chen Wang, Enxin Song, Jiatao Gu, Lingjie Liu
Abstract
Compositional 3D scene reconstruction has recently been explored from two directions: agentic reasoning that provides semantic understanding of spatial relationships but lacks precise alignment with input images; and visual geometry foundation models that predict dense point maps from input images but the reconstruction quality is limited. Therefore, recovering a complete 3D scene from a single monocular image with accurate inter-object relationships and high-fidelity reconstruction quality remains challenging. In this paper, we present HARMONY, a hierarchical chain-of-thought framework that leverages both agentic reasoning and visual geometry foundation. Given an image of an indoor scene, starting from an empty 3D floorplan, HARMONY first calibrates the camera against the reference image to establish a semantically-grounded spatial frame, then uses agentic VLM reasoning to recover the 3D room layout and an initial placement order. It then places the objects in a hierarchical order, from wall-mounted elements, free-standing furniture, to dependent decorations on top of furniture. We also use depth-first traversal for furniture so each placement conditions on previously resolved structure and a reflective feedback loop to avoid error accumulation. After each object placement by VLM, we use the point cloud estimations to perform geometry-based refinement so that the rendered image aligns better with the input. HARMONY can produce 3D scenes that are semantically consistent and perceptually aligned with the reference image, extending single-image compositional reconstruction to complex indoor scene images. Experiments on synthetic and real-world images demonstrate that HARMONY outperforms the evaluated reconstruction baselines, while qualitative comparisons with GPT-6 Astra suggest more faithful object arrangements and better preservation of scene details.