Mixed reality setup shows robots moving realistically in your space
Hardware-Free Robotics Laboratories in Mixed Reality
RoboticsHuman-Computer Interaction
Summary
Learning robotics usually relies on watching robots move on a flat computer screen, which doesn’t show the real size or space around the robot. The authors created MR-Robotics LAB, a tool that lets students see robot movements replayed in real size inside their physical environment using a VR headset. This system takes robot paths designed in MATLAB and plays them back with accurate positions and collisions in mixed reality on the Meta Quest 3 device. Students found the setup easy and felt it helped them understand how robot workspaces look in reality, though no direct comparison with traditional simulators was made.
What this means in practice
- •For engineering trainers: Provide hands-on robot movement visualization at real scale without needing physical hardware by replaying MATLAB-designed trajectories in mixed reality.
- •For virtual reality developers: Integrate physics-enabled robot trajectory playback in VR education apps to improve understanding of spatial robot behaviors using existing MATLAB data.
Authors
Santiago Berrezueta-Guzman, Habiba-Loai Khalil, Andrei Koshelev, Vanesa Metaj, Stefan Wagner
Abstract
Teaching robotics relies on screen-based simulation, showing robot motion in an abstract coordinate frame rather than at real scale in the learner's own space, while access to physical hardware is limited by cost, safety, and scheduling constraints. We present MR-Robotics LAB, a mixed-reality (MR) platform that replays MATLAB-generated robot trajectories at real scale within the learner's physical environment. A browser-based service validates a MATLAB workspace file (.mat), normalizes units, and publishes a versioned JSON trajectory; a Unity application on a Meta Quest 3 then reproduces the authored joint configurations under position control and replays them at the declared frame rate within a physics-enabled scene that supports collision detection and end-effector grasping. A formative single-group evaluation with engineering students found that participants reported low setup effort (M = 4.67 on a 5-point scale) and perceived support for workspace understanding from multi-viewpoint inspection (M = 4.56), and 83% of participants affirmed their willingness to use the platform in an introductory robotics course. The evaluation instrument records only perceived outcomes, without counterbalancing or a learning measure, so no comparative advantage over desktop simulation is claimed. The contribution is a reusable simulation-to-MR trajectory pathway and design guidance for hardware-free robot visualization in engineering education.