Physical interaction readiness lags in 3d indoor scene generation
ScIntBench: Physical Interaction Plausibility in 3D Scene Generation
Computer Vision and Pattern Recognition
Summary
Generating 3D indoor scenes that look correct and stable at rest is easier than making scenes that you can physically interact with realistically. The authors studied how well current 3D scene generators handle actions like pushing or opening objects in the scenes. They found many systems produce scenes that fail basic interaction tests despite appearing correct when static. This shows new challenges in creating 3D scenes that behave reliably when you move or manipulate things inside them.
What this means in practice
- •For virtual reality developers: Improve indoor 3D scene assets by verifying physical interaction capability during content creation workflows to enhance user experience.
- •For robotics simulation engineers: Select or refine 3D scene generation tools based on dynamic interaction tests to better simulate robot-object manipulation tasks indoors.
Authors
Wendi Ying, Daochang Liu, Leonid Sigal, Ajmal Saeed Mian
Abstract
Existing evaluations of text-conditioned 3D indoor scene generation largely emphasize semantic fidelity and physical validity at rest, leaving interaction-time physical reliability insufficiently assessed. To address this gap, we introduce ScIntBench, a unified benchmark centered on interaction readiness: it executes prescribed manipulation actions (e.g., pushing, opening, and surface probing) in physics simulation and measures task success, capability and progress, post-interaction integrity, and dynamic plausibility. Multi-view semantic and static physical evaluations complement these metrics and help explain interaction outcomes. We systematically evaluate 12 state-of-the-art scene generation systems across 10 room types. Our results reveal that strong semantic and static physical performance does not necessarily translate into interaction readiness: a system with near-perfect static plausibility completes only 11.54% of interactions, 7 of the 12 systems produce scenes that cannot support any interaction, and even the best system succeeds in only 36.92% of prescribed tasks. These findings highlight two fundamental challenges in generating interaction-ready scenes: providing the necessary physical interaction capabilities and ensuring reliable physical behavior under active manipulation.