Summary
Robots with legs make a lot of noise when they move, like footsteps and motor sounds, which can cover up other important sounds around them. The authors created a method that helps these robots pick out and remove their own noise from what they hear, without needing special recordings or labels. They do this by first finding parts of recordings with mostly robot noise, then mixing those with many environmental sounds to train a computer model. The model can then separate the robot's noise from any other sounds it might hear, even if those sounds are new or unknown. Tests on different walking robots show this approach improves the robot's hearing without extra manual work.
ego-noiseopen-set separationlegged robotsacoustic sensingunsupervised learningpretrained modelsaudio embeddingzero-shot learningsound event datasetneural separator
Authors
Koki Shoda, Jun Younes Louhi Kasahara, Aoba Koyanagi, Qi An, Atsushi Yamashita
Abstract
This paper proposes an open-set ego-noise separation framework for legged-robot audition via annotation-free adaptation and pretrained-model transfer. The framework removes robot-specific ego-noise while preserving environmental sounds whose classes are not specified in advance. Acoustic sensing provides cues about a robot's surroundings beyond the visual field, but walking-induced ego-noise from footstep impacts, joint-backlash rattling, and motor noise severely contaminates the recordings. The framework first uses RecurGraph to select ego-noise-dominant clips from the unlabeled recordings by aggregating clip embeddings into an embedding centroid and propagating scores over an audio-embedding graph. The selected clips are mixed with diverse environmental sounds from a large-scale sound-event dataset to provide paired mixture--target supervision for open-set separation. Transfer-DiT then adapts a general-purpose zero-shot neural separator to achieve high-fidelity open-set ego-noise separation for the target robot. Experiments with bipedal and quadrupedal robots show reliable clip selection and improvements in separation quality and downstream task performance over baseline separators. These results demonstrate the feasibility of annotation-free adaptation without separately recorded ego-noise-only data or manual clip-level annotations.