Scene retargeting transfers object layouts for realistic 3D interiors
Scene Retargeting: Learning Object Placement with Analogical Transfer
Computer Vision and Pattern Recognition
Summary
Making realistic 3D room layouts is hard because each space has its own shape and rules. The authors created a way to copy how objects are arranged in one room and apply that arrangement to a different room, even if the rooms have different sizes or objects. Their method groups objects and moves these groups while keeping the overall style and ensuring physical rules like walls and passageways stay correct. This approach works better than earlier methods on a large 3D room dataset and helps in tasks like creating virtual scenes from real ones or moving paths between rooms.
What this means in practice
- •For game developers: Automatically create new indoor 3D levels by adapting layouts from existing scenes with realistic object placement and clear walkways.
- •For augmented reality app developers: Generate virtual interiors that mimic real spaces for AR experiences, enabling consistent object arrangements adapted to new floor plans.
Authors
Minkwan Kim, Junho Kim, Seungmin Lee, Changwoon Choi, Young Min Kim
Abstract
Interactive simulations of embodied AI or spatial computing applications build on realistic 3D scenes that support daily activities. However, sparse, irregular layout structures impose scene-specific physical constraints, making it hard to define a generalizable framework for generating similar functional context. We formalize Scene Retargeting as stably transferring the semantically coherent spatial organization across layouts, rather than relying on textual descriptions or pairwise relationships. Our cluster-wise transfer flexibly handles mismatched object instances and adapts to distinctive floor plans. We optimize to preserve the rich semantic context of individual clusters by respecting the spatial distribution of foundation features. We can then impose physical constraints to refine wall contacts, pairwise alignment, or clear passageways and openings. Our framework outperforms state-of-the-art methods on layout generation on the 3D-FRONT dataset, and demonstrates downstream applications including real-to-sim transfer, analogical trajectory transfer, and multi-reference composition.