Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks

We present a novel learning-based approach to estimate the direction-of-arrival (DOA) of a sound source using a convolutional recurrent neural network (CRNN) trained via regression on synthetic data and Cartesian labels. We also describe an improved method to generate synthetic data to train the neural network using state-of-the-art sound propagation algorithms that model specular as well as diffuse reflections of sound. We compare our model against three other CRNNs trained using different formulations of the same problem: classification on categorical labels, and regression on spherical coordinate labels. In practice, our model achieves up to 43% decrease in angular error over prior methods. The use of diffuse reflection results in 34% and 41% reduction in angular prediction errors on LOCATA and SOFA datasets, respectively, over prior methods based on image-source methods. Our method results in an additional 3% error reduction over prior schemes that use classification based networks, and we use 36% fewer network parameters.

[1]  Michael S. Brandstein,et al.  A robust method for speech signal time-delay estimation in reverberant rooms , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  Cassia Valentini-Botinhao,et al.  Noisy speech database for training speech enhancement algorithms and TTS models , 2017 .

[3]  Jacob Benesty,et al.  Real-time passive source localization: a practical linear-correction least-squares approach , 2001, IEEE Trans. Speech Audio Process..

[4]  Michael J. Gerzon Periphony: With-Height Sound Reproduction , 1973 .

[5]  Thomas Fang Zheng,et al.  Noisy training for deep neural networks in speech recognition , 2015, EURASIP Journal on Audio, Speech, and Music Processing.

[6]  Archontis Politis,et al.  Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks , 2018, IEEE Journal of Selected Topics in Signal Processing.

[7]  Javier Macías Guarasa,et al.  Towards End-to-End Acoustic Localization Using Deep Learning: From Audio Signals to Source Position Coordinates , 2018, Sensors.

[8]  Patrick A. Naylor,et al.  The LOCATA Challenge Data Corpus for Acoustic Source Localization and Tracking , 2018, 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[9]  Dinesh Manocha,et al.  Improving Reverberant Speech Training Using Diffuse Acoustic Simulation , 2019, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Dinesh Manocha,et al.  Interactive Sound Propagation and Rendering for Large Multi-Source Scenes , 2016, ACM Trans. Graph..

[11]  Michael S. Brandstein,et al.  Robust Localization in Reverberant Rooms , 2001, Microphone Arrays.

[12]  Sanjeev Khudanpur,et al.  A study on data augmentation of reverberant speech for robust speech recognition , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Alan W. Black,et al.  The CMU Arctic speech databases , 2004, SSW.

[14]  Haizhou Li,et al.  A learning-based approach to direction of arrival estimation in noisy and reverberant environments , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Dinesh Manocha,et al.  High-order diffraction and diffuse reflections for interactive sound propagation in large environments , 2014, ACM Trans. Graph..

[16]  Sanjeev Khudanpur,et al.  Librispeech: An ASR corpus based on public domain audio books , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[18]  G. Carter,et al.  The generalized correlation method for estimation of time delay , 1976 .

[19]  Wen Zhang,et al.  Surround by Sound: A Review of Spatial Audio Recording and Reproduction , 2017 .

[20]  Emmanuel Vincent,et al.  CRNN-based Joint Azimuth and Elevation Localization with the Ambisonics Intensity Vector , 2018, 2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC).

[21]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[22]  Andrés Pérez-López,et al.  Ambisonics Directional Room Impulse Response as a New Convention of the Spatially Oriented Format for Acoustics , 2018 .

[23]  Dinesh Manocha,et al.  GSound: Interactive Sound Propagation for Games , 2011 .