Real to simulation framework improves policy evaluation accuracy
ReVeal: A Reconstruction-Aware Real-to-Sim Framework for VLA Policy Evaluation
RoboticsArtificial Intelligence
Summary
Testing robots or AI systems in the real world can be tricky and expensive, so people often test them in simulations instead. But if the simulated environment doesn't closely match the real world, the test results can be misleading. The authors created a system called ReVeal that helps check how well reconstructed environments for simulation match the real workspace. They introduce new ways to measure this match and improve the accuracy of these reconstructions. This makes simulated testing more reliable for vision and action tasks.
What this means in practice
- •For robotics developers: Evaluate robotic manipulation policies more accurately by verifying simulation environments against real-world reconstructions.
- •For augmented reality engineers: Improve the fidelity of reconstructed workspaces to better test interactive AR systems in virtual settings.
Authors
Xinyi Wang, Heng Hao, Wenjun Hu, Anna Enyu Li, Dizhi Ma, Karthik Ramani, Hankyu Moon, Yeong-Dae Kwon
Abstract
Simulation-based evaluation provides a scalable and repeatable alternative to real-world evaluation of vision-language-action (VLA) policies. However, reconstruction errors can cause simulated policy performance to diverge from real-world performance, motivating the need to assess reconstructed environments for downstream VLA policy evaluation. We present ReVeal, a real-to-sim assessment framework combining workspace reconstruction, reconstruction-level assessment, and matched closed-loop policy evaluation. Novel-View Mesh Fidelity (NVMF) and Annotated Planar Geometry Fidelity (APGF) assess observation and planar geometric fidelity, respectively. We also develop PGSR-D, a reconstruction pipeline incorporating monocular depth supervision to improve geometry where multi-view visual cues are limited. Across 8 assessment scenes, NVMF and APGF consistently distinguish the fidelity of 2DGS, PGSR, and PGSR-D. Matched evaluations of GR00T, SmolVLA, and pi0.5 across 8 humanoid manipulation tasks show consistent ordering between reconstruction fidelity and real-sim performance agreement across pipelines. Further analysis of the evaluation workspaces shows that higher fidelity is associated with stronger real-sim agreement.