Speaker localization and speech extraction with the EAR sensor

This paper presents the embedded audition for robotics (EAR) project internally developed at LAAS and its application to speaker localization and extraction. Hardware and software issues are first thoroughly depicted, concerning the development of an auditory sensor based on an array of microphones, a homemade dedicated acquisition chain and a FPGA based processing board. Then, the EAR sensor is assessed against various scenarios, in real noisy robotics environments. Localization results are presented when a speaker emits an utterance in the presence of a disturbing source. These validate the underlying theory and suggest further theoretical and experimental developments.

[1]  Horst-Michael Groß,et al.  Adaptive Noise Reduction and Voice Activity Detection for improved Verbal Human-Robot Interaction using Binaural Data , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[2]  Jean Rouat,et al.  Robust Recognition of Simultaneous Speech by a Mobile Robot , 2007, IEEE Transactions on Robotics.

[3]  Hideki Asoh,et al.  Sound source localization and signal separation for office robot "JiJo-2" , 1999, Proceedings. 1999 IEEE/SICE/RSJ. International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI'99 (Cat. No.99TH8480).

[4]  Philippe Souères,et al.  Modal Analysis Based Beamforming for Nearfield or Farfield Speaker Localization in Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  B.D. Van Veen,et al.  Beamforming: a versatile approach to spatial filtering , 1988, IEEE ASSP Magazine.

[6]  Thomas Kailath,et al.  Eigenstructure methods for direction of arrival estimation in the presence of unknown noise fields , 1986, IEEE Trans. Acoust. Speech Signal Process..

[7]  Hiroshi Mizoguchi,et al.  Three ring microphone array for 3D sound localization and separation for mobile robot audition , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Hong Wang,et al.  Coherent signal-subspace processing for the detection and estimation of angles of arrival of multiple wide-band sources , 1985, IEEE Trans. Acoust. Speech Signal Process..

[9]  Patrick Danès,et al.  Convex optimization and modal analysis for beamforming in robotics: Theoretical and implementation issues , 2007, 2007 15th European Signal Processing Conference.

[10]  Jean-Yves Fourniols,et al.  REAL-TIME STEREOVISION BY AN INTEGRATED SENSOR , 2007 .

[11]  Thushara D. Abhayapala,et al.  Range and bearing estimation of wideband sources using an orthogonal beamspace processing structure , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[12]  Philippe Souères,et al.  An experimental testbed for sound source localization with mobile robots using optimized wideband beamformers , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Stefan Wermter,et al.  Bioinspired Auditory Sound Localisation for Improving the Signal to Noise Ratio of Socially Interactive Robots , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  A. Lee Swindlehurst,et al.  A Performance Analysis ofSubspace-Based Methods in thePresence of Model Errors { Part I : The MUSIC AlgorithmA , 1992 .

[15]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[16]  Hideharu Amano,et al.  Implementation of active direction-pass filter on dynamically reconfigurable processor , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Munsang Kim,et al.  Probabilistic Speaker Localization in Noisy Environments by Audio-Visual Integration , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Patrick Danès,et al.  Broadband variations of the MUSIC high-resolution method for Sound Source Localization in Robotics , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Hiroshi G. Okuno,et al.  An open source software system for robot audition HARK and its evaluation , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.