Livin benchmark tests robot skills in real home setups
LIVIN: Benchmarking Spatial and Embodied Intelligence in Digital Twins of Lived-In Homes
Computer Vision and Pattern Recognition
Summary
Robots need homes to look and feel real for them to understand and interact well. The authors created LIVIN, a set of 30 digital home copies that show exactly how real homes are organized with furniture and belongings. They made sure every detail matched the real homes by having humans check the digital versions at each step. LIVIN tests robots on tasks like seeing objects, building 3D maps, moving around, and handling items inside these homes. The authors found that current robots still struggle with cluttered spaces and practical constraints found in real houses.
What this means in practice
- •For robotics developers: Test and improve robot sensing and movement inside realistic home environments using detailed replicas of actual lived-in homes.
- •For home automation engineers: Design home robots that better navigate and interact amidst common household furniture and clutter by using spatial layouts from real homes.
Authors
Peijun Xu, Chuansen Nie, Yiyang He, Yinuo Bai, Jingyang Liu, Kuixiang Shao, Yuyang Jiao, Kuanhao Xia, Jiayi Zhu, Zitian Yang, Yanqi Zhang, Tianye Tan, Shuwei Di, Junyi Xu, Jingyi Yu, Jiayuan Gu
Abstract
Realistic household simulation must capture not only diverse environments but also the lived-in object arrangements and spatial constraints that shape robot motion and interaction. Existing resources often trade off scale, real-world correspondence, and interaction readiness, leaving a gap in faithful, interactive replicas of how real homes are actually arranged. To this end, we introduce LIVIN, a benchmark for spatial and embodied intelligence built on digital twins of 30 diverse lived-in homes. These replicas preserve observed room layouts, furniture configurations, and everyday belongings. To construct them, we design a human-in-the-loop workflow comprising instance recognition, architectural reconstruction, and object generation and placement, with intermediate results reviewed and corrected by humans against the source observations at each stage. We evaluate four tasks in LIVIN: 3D detection, 3D reconstruction, navigation, and loco-manipulation. Our evaluations show that current methods remain challenged by the dense object arrangements, occlusions, limited free space, and constrained interaction regions found in realistic lived-in homes. We hope LIVIN will help advance embodied AI in real-world homes, from spatial understanding to robotic interaction, and ultimately bring embodied intelligence into everyday home environments.