Papers for

robot simulation developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

New method converts photorealistic 3D scenes into interactive environments

φ-RIE: From Photorealistic Reconstruction to Interactive Environments

Abstract: 3D Gaussian Splatting (3DGS) can reconstruct a captured scene photorealistically, but the resulting representation does not by itself support physical interaction. Robot simulation instead requires object-level change, \textit{i.e.}, objects must move independently, make contact, and reveal previously occluded surroundings. This gap arises because object appearance may remain entangled with the background, while hidden object geometry and occluded background content may be unobserved. To address this challenge, we present φ-RIE, a Gaussian-native pipeline that converts selected objects into movable simulator assets while preserving the remaining reconstruction. Our key observation is that asset construction and source removal should be coupled, \textit{i.e.}, one object identity should define the movable asset and the scene content to remove and complete. Accordingly, Scene Observation supplies shared evidence to Coupled Scene Construction, which creates registered assets and completed background Gaussians for simulator-driven rendering in an Interactive Environment. This coupling preserves unedited Gaussians while aligning visual and physical state. On 50 ScanNet++ scenes, evidence-based selection and registration retry increase matched F1 at 20\,mm from 0.336 to 0.383 at fixed retention. Further tests demonstrate asset executability, manipulation gains over a single-generator baseline, and the visual cost of conversion. Together, these results demonstrate that \name\ enables interactive scene conversion.

Tue 22 SeptRoboticsComputer Vision and Pattern RecognitionGraphics
The gist
Photorealistic 3D reconstructions of scenes often don’t allow objects to be moved or interacted with separately, which is needed for robot simulations. The authors developed φ-RIE, a technique that turns parts of these 3D scenes into movable objects while keeping the rest of the scene intact and looking realistic. This lets virtual robots or simulations physically interact with objects like in the real world. Their results show improved matching and manipulation of objects within these interactive environments.
Open 2609.26795v1