Difference-Based Relational Learning for Zero-Shot Object-Goal Visual Navigation With Direct Sim-to-Real Transfer
Robotics
Summary
The authors proposed a new method called Temporal Difference-Relational Network (T-DRN) to help robots find objects using only visual input without needing retraining for real-world conditions. Their method compares differences between target and seen objects to create features that work well despite changes in appearance or limited camera views. They tested T-DRN in a simulation environment and on a real robot, showing it improved success rates and worked reliably without extra adjustments. This suggests their approach can transfer learning directly from simulation to real robots more effectively.
Authors
Guolei Qi, Feitian Zhang
Abstract
End-to-end deep reinforcement learning (DRL) for zero-shot object-goal visual navigation remains challenged by the sim-to-real gap, particularly variations in object appearance and restricted camera field-of-view (FoV). This letter proposes a Temporal Difference-Relational Network (T-DRN) for robust zero-shot sim-to-real transfer. T-DRN combines a Siamese difference-based feature extractor, which computes relational difference between the target and observed objects to produce domain-independent representations, with a dual-frame temporal buffer that preserves short-term object continuity under narrow FoV. Extensive experiments in AI2-THOR demonstrate that T-DRN improves zero-shot generalization in terms of success rates over strong baselines. Furthermore, T-DRN is systematically validated on a physical wheeled robot, demonstrating robust performance under real sensing and actuation constraints and supporting the feasibility of direct sim-to-real transfer.