Robot uses sound and depth to navigate toward hidden sources
BCNav: Bearing-Conditioned Depth Policies for Sound Source Navigation
Robotics
Summary
Finding the direction of a sound helps robots move toward it even when they can't see it. The authors created a method where a robot uses a sound direction estimate and depth camera images to decide how to move, without needing expensive sound simulations or detailed maps. They trained the robot to follow the best path using examples, and it works both in computer tests and on real robots without extra tuning. This approach makes it easier for robots to respond to sounds in new places.
What this means in practice
- •For robot developers: Equip robots to locate and approach sound sources in unknown environments using only depth images and bearing estimates without acoustic simulator tuning.
- •For security system integrators: Deploy security robots that move toward unexpected sounds without pre-mapping or specialized audio data collection.$Commercial implications: Enables selling security robots that autonomously investigate sounds with fewer setup demands due to decoupled audio processing and navigation control.
Authors
Yaozhong Kang, Jiang Wang, Takeshi Ashizawa, Benjamin Yen, Kazuhiro Nakadai
Abstract
The ability to navigate toward sound sources extends a robot's reach beyond its visual field, enabling response to auditory events in unknown environments. To equip robots with this capability, existing methods couple acoustic and visual information through joint audio-visual learning in acoustic simulators. However, acoustic simulation is both low-fidelity and expensive, producing a domain gap that prevents reliable real-world deployment, while the discrete action spaces inherited from grid-based simulators introduce an additional kinematic gap on physical robots. To alleviate these issues, we propose BCNav, a decoupled framework that separates the acoustic module from the learned navigation policy using direction-of-arrival (DOA) estimation: an estimator provides a scalar bearing to the sound source, so the navigation policy only processes depth images and a bearing angle, two inputs whose domain gaps are well characterized. We collect shortest-path demonstrations with calibrated bearing noise injection and train the policy via imitation learning to output continuous velocity commands directly executable on ground robots. We demonstrate the method in simulation and on a physical robot, navigating unknown environments without any acoustic fine-tuning, prior mapping, or real-world audio data collection. Code is available at https://github.com/york1to/bcnav.