A real-time 3D sound localization system with miniature microphone array for virtual reality

This paper presents a real-time three-dimensional (3D) wideband sound localization system designed with a miniature XYZO microphone array. Unlike the conventional microphone arrays for sound localization using only omnidirectional microphones, the presented microphone array is designed with both bidirectional (pressure gradient) and omnidirectional microphones. Therefore, the array has significantly reduced size and is known as the world's smallest microphone array design for 3D sound source localization in air. In this paper, we describe the 3D array configuration and perform array calibration. For 3D sound localization, we provide studies on the array output model of the XYZO array, the widely known direction-of-arrival (DOA) estimation methods and the direction search in 3D space. To achieve the real-time processing for 1° search resolution, we accelerate the parallel computations on GPU platform with CUDA programming, and a 130X speedup is achieved compared to a multi-thread CPU implementation. The performance of the proposed system is studied under various reverberation lengths and signal-to-noise levels. We also demonstrate a real-time 3D sound localization demo showing good ability to virtual reality.

[1]  M. Hawkes,et al.  Hull-mounted acoustic vector-sensor processing , 1995, Conference Record of The Twenty-Ninth Asilomar Conference on Signals, Systems and Computers.

[2]  Anoop Gupta,et al.  Distributed meetings: a meeting capture and broadcasting system , 2002, MULTIMEDIA '02.

[3]  Douglas L. Jones,et al.  Beamformer performance with acoustic vector sensors in air. , 2006, The Journal of the Acoustical Society of America.

[4]  J. Capon.  High-resolution frequency-wavenumber spectrum analysis , 1969 .

[5]  Alex Pentland,et al.  Smart headphones: enhancing auditory awareness through robust speech detection and source localization , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[6]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[7]  Joseph H. DiBiase.  A High-Accuracy, Low-Latency Technique for Talker Localization in Reverberant Environments Using Microphone Arrays , 2000 .

[8]  Hong Wang,et al.  Voice source localization for automatic camera pointing system in videoconferencing , 1997, Proceedings of 1997 Workshop on Applications of Signal Processing to Audio and Acoustics.

[9]  Douglas L. Jones,et al.  Localization of multiple acoustic sources with small arrays using a coherence test. , 2008, The Journal of the Acoustical Society of America.