Development of Robot Audition under Severe Conditions

The ability of robots to listen to several things at once with their own “ears”, i.e., robot audition, is critical in improving the performance of search and rescue activities under severe conditions. This paper introduces “HARK” robot audition open-source software and its capabilities of suppressing ego-noise that is caused by robot’s own movements such as motor, propeller and/or flying noise. Then it describes three main applications of robot audition: 1) Unmanned Aerial Vehicle (UAV) with a microphone array to capture sounds can localize a sound source by suppressing ego-noise with either hovering, slow gliding or fast gliding. It can also recognize a sound source by CNN. 2) A serpentine robot with a microphone array can estimate its posture by sound. It can also enhance a voice by Online Robust PCA. 3) A robot with a LiDAR and 32-channel microphone can visualize a sound map by superimposing sound source directions on point clouds.