Embodied Scene Rearrangement Planning
Robotics
Summary
The authors introduce a new challenge called Embodied Scene Rearrangement Planning (ESRP), where robots rearrange furniture in 3D rooms to match a target layout but only see from their own viewpoint, without full knowledge of the room. This makes the task harder because furniture can block the robot's view and it has to plan many steps ahead with limited information. They created ESRP-Bench, a large dataset and benchmark with thousands of scenes to test different approaches. Their experiments show that current methods still find this task very difficult, pointing to future work needed for robots to better understand and manipulate real environments.
Authors
Canzhi Chen, Zan Wang, Siqi Zhu, Qi Wu, Yixuan Li, Wei Liang
Abstract
This paper introduces Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied agents to rearrange furniture in 3D scenes to match a target configuration using only egocentric observations and a top-down target layout. Unlike prior rearrangement tasks, ESRP precludes global state access and introduces mutual object occlusions, reflecting the practical constraints of real-world robotic deployment. These factors make aligning partial egocentric observations with the global target layout particularly challenging for long-horizon planning. To facilitate research, we present ESRP-Bench, a comprehensive benchmark built on OmniGibson featuring over 5,400 scene pairs and 8,200 objects. We define three multi-level metrics to evaluate rearrangement quality and provide four baselines: a hierarchical task-and-motion planning method, a vision-language-model-based method, and two learning-based approaches (IL and RL). Experimental results demonstrate that current methods struggle to complete the task efficiently, highlighting ESRP as a challenging frontier for embodied agents in scene understanding and long-horizon task planning. This work serves as a stepping stone toward deploying intelligent agents in real-world scenarios. Project page: https://pie-lab.cn/ESRP/.