XCycles Backprojection Acoustic Super-Resolution

The computer vision community has paid much attention to the development of visible image super-resolution (SR) using deep neural networks (DNNs) and has achieved impressive results. The advancement of non-visible light sensors, such as acoustic imaging sensors, has attracted much attention, as they allow people to visualize the intensity of sound waves beyond the visible spectrum. However, because of the limitations imposed on acquiring acoustic data, new methods for improving the resolution of the acoustic images are necessary. At this time, there is no acoustic imaging dataset designed for the SR problem. This work proposed a novel backprojection model architecture for the acoustic image super-resolution problem, together with Acoustic Map Imaging VUB-ULB Dataset (AMIVU). The dataset provides large simulated and real captured images at different resolutions. The proposed XCycles BackProjection model (XCBP), in contrast to the feedforward model approach, fully uses the iterative correction procedure in each cycle to reconstruct the residual error correction for the encoded features in both low- and high-resolution space. The proposed approach was evaluated on the dataset and showed high outperformance compared to the classical interpolation operators and to the recent feedforward state-of-the-art models. It also contributed to a drastically reduced sub-sampling error produced during the data acquisition.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Jung-Woo Choi,et al.  Direction of arrival estimation using nonsingular spherical ESPRIT. , 2018, The Journal of the Acoustical Society of America.

[3]  Tao Zhang,et al.  A Modified Frequency Weighted MUSIC Algorithm for Multiple Sound Sources Localization , 2018, 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP).

[4]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[5]  Kishor P. Upla,et al.  Thermal Image Super-Resolution Challenge - PBVS 2021 , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Bruno da Silva,et al.  CABE: A Cloud-Based Acoustic Beamforming Emulator for FPGA-Based Sound Source Localization , 2019, Sensors.

[7]  Henrique S. Malvar,et al.  A new beamformer design algorithm for microphone arrays , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[8]  Douglas L. Maskell,et al.  The estimation of subsample time delay of arrival in the discrete-time measurement of phase delay , 1999, IEEE Trans. Instrum. Meas..

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Kishor P. Upla,et al.  Thermal Image Super-Resolution Challenge - PBVS 2020 , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Vamsynagh Pedamallu Microphone Array Wiener Beamforming with emphasis on Reverberation , 2012 .

[14]  Marc Moonen,et al.  Generalized sidelobe canceller based combined acoustic feedback- and noise cancellation , 2008, Signal Process..

[15]  Prasanga N. Samarasinghe,et al.  Sound Source Localization in a Reverberant Room Using Harmonic Based Music , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Christoph Studer,et al.  FPGA-based real-time acoustic camera prototype , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[17]  Ramani Duraiswami,et al.  Accelerated speech source localization via a hierarchical search of steered response power , 2004, IEEE Transactions on Speech and Audio Processing.

[18]  Bruno da Silva,et al.  A Multimode SoC FPGA-Based Acoustic Camera for Wireless Sensor Networks , 2018, 2018 13th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC).

[19]  Pascal Scalart,et al.  Computationally efficient and robust frequency-domain GSC , 2010 .

[20]  Nele Mentens,et al.  AITIA: Embedded AI Techniques for Embedded Industrial Applications , 2020, 2020 International Conference on Omni-layer Intelligent Systems (COINS).

[21]  An Braeken,et al.  M3-AC: A Multi-Mode Multithread SoC FPGA Based Acoustic Camera , 2021, Electronics.

[22]  Unto K. Laine,et al.  Splitting the unit delay [FIR/all pass filters design] , 1996, IEEE Signal Process. Mag..

[23]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[24]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Lara del Val Puente,et al.  Design and Evaluation of a Scalable and Reconfigurable Multi-Platform System for Acoustic Imaging , 2016, Sensors.

[26]  Federico Domínguez,et al.  SoundCompass: A Distributed MEMS Microphone Array-Based Sensor for Sound Source Localization , 2014, Sensors.

[27]  Rafael E. Rivadeneira,et al.  Thermal Image Super-resolution: A Novel Architecture and Dataset , 2020, VISIGRAPP.

[28]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[30]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Hervé Bourlard,et al.  Microphone array beampattern characterization for hands-free speech applications , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[32]  Fahad Shahbaz Khan,et al.  AIM 2020 Challenge on Real Image Super-Resolution: Methods and Results , 2020, ECCV Workshops.

[33]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  James R. Glass,et al.  SVD-PHAT: A Fast Sound Source Localization Method , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[35]  Jonathon Shlens,et al.  A Learned Representation For Artistic Style , 2016, ICLR.

[36]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Olivier Debeir,et al.  Multimodal Sensor Fusion In Single Thermal image Super-Resolution , 2018, ACCV Workshops.

[38]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[39]  Qinghua Huang,et al.  Direction of Arrival Estimation Using Distributed Circular Microphone Arrays , 2018, 2018 14th IEEE International Conference on Signal Processing (ICSP).

[40]  Qiang Chen,et al.  Network In Network , 2013, ICLR.