Multiview Soundfield Imaging in the Projective Ray Space

A soundfield image is a data structure that efficiently encodes and represents the wave field as captured by a microphone array. Its representation is based on the directional plenacoustic function, which is defined as the radiance of the acoustic paths (rays) that cross the segment that the array lies upon. The soundfield image can be processed “as is” to develop a variety of applications. In its original formulation, the soundfield image is based on a Euclidean parameterization that can accommodate a limited range of rays and is suitable for managing a single array only. In this paper, we generalize this methodology to the case of multiple microphone arrays deployed in space. The use of multiple arrays allows us to capture truly global information on the sound field but requires us to rethink the ray space, and adopt a global representation of the acoustic rays based on projective geometry. After introducing the new parameterization, we present two examples of applications: the estimation of the mutual poses of two or more arrays (self-calibration); and the localization of multiple acoustic sources. The effectiveness of these applications is proven through simulations as well as real data experiments.

[1]  Augusto Sarti,et al.  Acoustic Source Localization With Distributed Asynchronous Microphone Networks , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[3]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  Maurizio Omologo,et al.  Acoustic event localization using a crosspower-spectrum phase based technique , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Walter Kellermann,et al.  On 2D localization of reflectors using robust beamforming techniques , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Maximo Cobos,et al.  A Modified SRP-PHAT Functional for Robust Real-Time Sound Source Localization With Scalable Spatial Sampling , 2011, IEEE Signal Processing Letters.

[7]  Augusto Sarti,et al.  From direction of arrival estimates to localization of planar reflectors in a two dimensional geometry , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[9]  Filippo Maria Fazi,et al.  Representation of sound fields for audio recording and reproduction , 2012 .

[10]  Jorge Stolfi,et al.  Oriented Projective Geometry: A Framework for Geometric Computations , 2014 .

[11]  Augusto Sarti,et al.  Soundfield Imaging in the Ray Space , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[12]  Edward H. Adelson,et al.  Single Lens Stereo with a Plenoptic Camera , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Mikael Mieskolainen,et al.  Closed-form self-localization of asynchronous microphone arrays , 2011, 2011 Joint Workshop on Hands-free Speech Communication and Microphone Arrays.

[14]  Darren B. Ward,et al.  Particle filter beamforming for acoustic source localization in a reverberant environment , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[15]  Heinrich Kuttruff,et al.  Room acoustics , 1973 .

[16]  Ramani Duraiswami,et al.  Plane-Wave Decomposition of Acoustical Scenes Via Spherical and Cylindrical Microphone Arrays , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[17]  Gary W. Elko,et al.  A highly scalable spherical microphone array based on an orthonormal decomposition of the soundfield , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  Augusto Sarti,et al.  Tracking multiple acoustic sources using particle filtering , 2006, 2006 14th European Signal Processing Conference.

[19]  Holography Book,et al.  Fourier Acoustics Sound Radiation And Nearfield Acoustical Holography , 2016 .

[20]  Augusto Sarti,et al.  Tracking Multiple Acoustic Sources in Reverberant Environments using Regularized Particle Filter , 2007, 2007 15th International Conference on Digital Signal Processing.

[21]  Sakari Tervo,et al.  Acoustic reflection localization from room impulse responses , 2012 .

[22]  R. Kress,et al.  Inverse Acoustic and Electromagnetic Scattering Theory , 1992 .

[23]  T. Ajdler,et al.  The Plenacoustic Function and Its Sampling , 2006, IEEE Transactions on Signal Processing.

[24]  Andrea Ridolfi,et al.  On a Stochastic Version of the Plenacoustic Function , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[25]  Richard Heusdens,et al.  Auto-localization in ad-hoc microphone arrays , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Augusto Sarti,et al.  REAL TIME MODELING OF ACOUSTIC PROPAGATION IN COMPLEX ENVIRONMENTS , 2004 .

[27]  Emanuel A. P. Habets,et al.  Inference of Room Geometry From Acoustic Impulse Responses , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[28]  Martin Vetterli,et al.  Acoustic echoes reveal room shape , 2013, Proceedings of the National Academy of Sciences.

[29]  Augusto Sarti,et al.  Geometric reconstruction of the environment from its response to multiple acoustic emissions , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]  Christian Schindelhauer,et al.  Network synchronization and localization based on stolen signals , 2011, PODC '11.

[31]  Walter Kellermann,et al.  Joint DOA and TDOA estimation for 3D localization of reflective surfaces using eigenbeam MVDR and spherical microphone arrays , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Augusto Sarti,et al.  Soundfield rendering with loudspeaker arrays through multiple beam shaping , 2009, 2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.

[33]  F. Alton Everest,et al.  Master handbook of acoustics , 1981 .

[34]  Jorge Stolfi,et al.  Oriented projective geometry , 1987, SCG '87.

[35]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[36]  Augusto Sarti,et al.  Fast Tracing of Acoustic Beams and Paths Through Visibility Lookup , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[37]  Gernot A. Fink,et al.  Towards acoustic self-localization of ad hoc smartphone arrays , 2011, 2011 Joint Workshop on Hands-free Speech Communication and Microphone Arrays.

[38]  Walter Kellermann,et al.  Acoustic source detection and localization based on wavefield decomposition using circular microphone arrays , 2006 .

[39]  Augusto Sarti,et al.  Rendering of directional sources through loudspeaker arrays based on plane wave decomposition , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[40]  Lilian Aveneau,et al.  Exact, robust and efficient full visibility computation in Plücker space , 2007, The Visual Computer.

[41]  Yves Grenier,et al.  Sound Field Analysis Based on Analytical Beamforming , 2007, EURASIP J. Adv. Signal Process..

[42]  Eric A. Lehmann,et al.  Particle Filter Design Using Importance Sampling for Acoustic Source Localisation and Tracking in Reverberant Environments , 2006, EURASIP J. Adv. Signal Process..

[43]  Ying Yu,et al.  A Real-Time SRP-PHAT Source Location Implementation using Stochastic Region Contraction(SRC) on a Large-Aperture Microphone Array , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.