PRISM: Personalized Robotic Dataset Generation via Image-based Scene and Motion Synthesis
2026-07-06 • Robotics
Robotics
AI summaryⓘ
The authors developed PRISM, a method that helps robots learn tasks better in specific user environments using just one image and a simple instruction. PRISM creates digital versions of real scenes that match the user's environment but still have variety, generating example robot actions without needing people to control the robot manually. Their tests showed that robots trained with PRISM data performed better on standard tasks and real-world manipulations, especially when tested in new or different environments. This approach reduces the need for costly and hard-to-scale teleoperation data collection.
vision-language-action modelsrobot policy learningteleoperationsimulationdigital twindataset synthesisrobot manipulationdomain generalizationnatural language instructionsrobot training
Authors
Dogyu Ko, Haneul Kim, Chanyoung Yeo, Dowoon Lee, Taeho Park, Hyoseok Hwang
Abstract
Recent advances in large-scale pretrained vision-language-action models have improved robot policy learning, but directly deploying such policies in user-specific environments remains challenging due to limited generalization, which inevitably requires collecting a dataset tailored to the target environment. Teleoperation yields well-aligned data but is costly and difficult to scale, whereas simulation scales easily but struggles to resemble the target environment and generate task-specific trajectories. To meet both simultaneously, we propose PRISM, an end-to-end pipeline that generates personalized robotic datasets from a single image and a natural-language instruction. PRISM constructs digital cousin scenes that are semantically and geometrically aligned with the user environment yet diverse at the instance level, and synthesizes executable demonstrations without human teleoperation. Extensive experiments show that policies trained on PRISM-generated datasets outperform those trained on baseline-generated datasets on LIBERO and LIBERO-Plus, achieve up to 100\% success rate on three real-world manipulation tasks, and maintain stronger performance when evaluated in environments that differ from those seen during training.