A sound source reconstruction approach based on the machine vision and inverse patch transfer functions method

Abstract The inverse patch transfer functions (iPTF) method can realize local reconstruction of a vibrating structure in non-anechoic environments, and the geometrical shape of the reconstruction area could be irregular as long as it is known. However, the shape of the non-planar reconstruction area is not easy to be obtained in practice, which is adverse to its application in the situ tests. To achieve the sound source reconstruction of the local area with unknown shape in non-anechoic environments, a hybrid method that combines the machine vision and the iPTF method is proposed. The machine vision technology is applied to facilitate the boundary modeling of a target radiation segment. It makes use of the BundleFusion method with an RGB-D camera to accomplish the 3D reconstruction of the target area, as well as the localization of microphones. A virtual cavity consisting of the target area, virtual measurement surface, and the gap between them could be constructed automatically. Consequently, the impedance matrix of the virtual cavity is obtained based on the boundary integral equation with the adoption of the free field Green’s function. A microphone array mounted up with a rigid masker is applied in the iPTF method, which can form the rigid Neumann boundary condition on the masker. A simulation is first carried out to investigate the influence of 3D reconstruction errors of the machine vision technology on the sound source reconstruction. Then, two experiments are performed to validate the feasibility and effectiveness of the reconstruction in noisy environments. One is conducted in the semi-anechoic room with a cylindrical radiator, and a loudspeaker is put aside as the interference source. The other is conducted in the cabin of an aircraft under cruising condition. It is demonstrated that the proposed method can realize the normal velocity reconstruction without prior geometry knowledge of the target area, and the magnitude and distribution of the dominant sound source could be reconstructed accurately.

[1]  Matthias Nießner,et al.  BundleFusion , 2016, TOGS.

[2]  Haijun Wu,et al.  A Collocation BEM for 3D Acoustic Problems Based on a Non-singular Burton-Miller Formulation With Linear Continuous Elements , 2018 .

[3]  Ding-Yu Hu,et al.  Identification of active sources inside cavities using the equivalent source method-based free-field recovery technique , 2015 .

[4]  Bing Li,et al.  Video visualization for moving sound sources based on binocular vision and short-time beamforming , 2010 .

[5]  Dimitris Samaras,et al.  EyeOpener: Editing Eyes in the Wild , 2017, ACM Trans. Graph..

[6]  A. Tikhonov,et al.  Numerical Methods for the Solution of Ill-Posed Problems , 1995 .

[7]  S. Yoshikawa,et al.  Reduction methods of the reconstruction error for large-scale implementation of near-field acoustical holography. , 2001, The Journal of the Acoustical Society of America.

[8]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[9]  Manuel Melon,et al.  Hemispherical double-layer time reversal imaging in reverberant and noisy environments at audible frequencies. , 2015, The Journal of the Acoustical Society of America.

[10]  Hao Jiang,et al.  Reconstructing the normal velocities of acoustic sources in noisy environments using a rigid microphone array. , 2016, The Journal of the Acoustical Society of America.

[11]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[12]  Paolo Castellini,et al.  Average acoustic beamforming in car cabins: An automatic system for acoustic mapping over 3D surfaces , 2018 .

[13]  Finn Jacobsen,et al.  Sound field reconstruction using acousto-optic tomography. , 2012, The Journal of the Acoustical Society of America.

[14]  Martin Bach-Andersen,et al.  Array based measurement of radiated and absorbed sound intensity components , 2008 .

[15]  Gian Marco Revel,et al.  A new laser vibrometry-based 2D selective intensity method for source identification in reverberant fields: part I. Development of the technique and preliminary validation , 2010 .

[16]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[17]  Haijun Wu,et al.  An inverse patch transfer functions method based on a free field Green's function , 2020 .

[18]  Junxian Wang,et al.  A sample of active galactic nuclei with strong soft X-ray variabilities , 2015 .

[19]  Hans-Peter Seidel,et al.  Interactive by-example design of artistic packing layouts , 2013, ACM Trans. Graph..

[20]  Thomas Padois,et al.  Application of acoustic imaging techniques on snowmobile pass-by noise. , 2017, The Journal of the Acoustical Society of America.

[21]  Yi-Chun Du,et al.  A Real-Time Speech Separation Method Based on Camera and Microphone Array Sensors Fusion Approach , 2020, Sensors.

[22]  Brian H Houston,et al.  Fast Fourier transform and singular value decomposition formulations for patch nearfield acoustical holography. , 2003, The Journal of the Acoustical Society of America.

[23]  Yong-Bin Zhang,et al.  Reconstruction of the sound field above a reflecting plane using the equivalent source method , 2017 .

[24]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[25]  Ding-Yu Hu,et al.  Reconstruction of the free-field radiation from a vibrating structure based on measurements in a noisy environment. , 2013, The Journal of the Acoustical Society of America.

[26]  C. Bi,et al.  Patch nearfield acoustic holography combined with sound field separation technique applied to a non-free field , 2015 .

[27]  E. Williams.  Continuation of acoustic near-fields. , 2003, Journal of the Acoustical Society of America.

[28]  Mathew Legg,et al.  Automatic 3D scanning surface generation for microphone array acoustic imaging , 2014 .

[29]  Jean-Louis Guyader,et al.  Identification of source velocities with Inverse Patch Transfer Functions method , 2008 .

[30]  Peter C. Y. Chen,et al.  Design and Evaluation of a Prototype System for Real-Time Monitoring of Vehicle Honking , 2019, IEEE Transactions on Vehicular Technology.

[31]  G. F. Miller,et al.  The application of integral equation methods to the numerical solution of some exterior boundary-value problems , 1971, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[32]  Enrico Primo Tomasini,et al.  An international review of laser Doppler vibrometry: Making light work of vibration measurement , 2017 .

[33]  S H Park,et al.  Visualization of pass-by noise by means of moving frame acoustic holography. , 2001, The Journal of the Acoustical Society of America.

[34]  Equivalent source model from acousto-optic measurements and application to an acoustic pulse characterization , 2019, Journal of Sound and Vibration.

[35]  Christophe Langrenne,et al.  Evaluation of a separation method for source identification in small spaces. , 2013, The Journal of the Acoustical Society of America.

[36]  Jean-Louis Guyader,et al.  Identification of source velocities on 3D structures in non-anechoic environments: Theoretical background and experimental validation of the inverse patch transfer functions method , 2010 .

[37]  Earl G Williams,et al.  Approximations of inverse boundary element methods with partial measurements of the pressure field. , 2008, The Journal of the Acoustical Society of America.