Legged robots separate walking noise from environment sounds without labels
Open-Set Ego-Noise Separation for Legged-Robot Audition via Annotation-Free Adaptation and Pretrained-Model Transfer
Robotics
Summary
Legged robots have trouble hearing what’s around them because their footsteps and motors make lots of noise. The researchers created a way for robots to learn and remove their own walking noises from sound recordings without needing people to label the noise beforehand. They use a method that finds parts of recordings mostly made up of robot noise and then teach a sound separation model using those plus extra environmental sounds. This approach works well on robots with different numbers of legs and improves their ability to understand sounds in noisy situations.
What this means in practice
- •For robotics engineers: Remove legged robots’ self-generated noise from audio to improve environment sound sensing without manual noise annotation.
- •For audio signal processing teams: Adapt general-purpose sound separation models to new robot noise types without labeled noise data.
Authors
Koki Shoda, Jun Younes Louhi Kasahara, Aoba Koyanagi, Qi An, Atsushi Yamashita
Abstract
This paper proposes an open-set ego-noise separation framework for legged-robot audition via annotation-free adaptation and pretrained-model transfer. The framework removes robot-specific ego-noise while preserving environmental sounds whose classes are not specified in advance. Acoustic sensing provides cues about a robot's surroundings beyond the visual field, but walking-induced ego-noise from footstep impacts, joint-backlash rattling, and motor noise severely contaminates the recordings. The framework first uses RecurGraph to select ego-noise-dominant clips from the unlabeled recordings by aggregating clip embeddings into an embedding centroid and propagating scores over an audio-embedding graph. The selected clips are mixed with diverse environmental sounds from a large-scale sound-event dataset to provide paired mixture--target supervision for open-set separation. Transfer-DiT then adapts a general-purpose zero-shot neural separator to achieve high-fidelity open-set ego-noise separation for the target robot. Experiments with bipedal and quadrupedal robots show reliable clip selection and improvements in separation quality and downstream task performance over baseline separators. These results demonstrate the feasibility of annotation-free adaptation without separately recorded ego-noise-only data or manual clip-level annotations.