Coupling Rendering and Generative Adversarial Networks for Artificial SAS Image Generation

There is a growing demand for large-scale Synthetic Aperture Sonar (SAS) datasets. This demand stems from data-driven applications such as Automatic Target Recognition (ATR) [1]–[3], segmentation [4] and oceanographic research of the seafloor, simulation for sensor prototype development and calibration [5], and even potential higher level tasks such as motion estimation [6] and micronavigation [7]. Unfortunately, the acquisition of SAS data is bottlenecked by the costly deployment of SAS imaging systems, and even when data acquisition is possible, the data is often skewed towards containing barren seafloor rather than objects of interest. This skew introduces a data imbalance problem wherein a dataset can have as much as a 1000-to-1 ratio of seafloor background to object-of-interest SAS image chips.

[1]  Daniel Köhntopp.  Shape-based Machine Perception of Man-Made Objects on Underwater Sensor Data , 2018 .

[2]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[3]  Yoichi Ochiai,et al.  GANs-based Clothes Design: Pattern Maker Is All You Need to Design Clothing , 2019, AH.

[4]  Jason E. Summers,et al.  Deep neural networks for learning classification features and generative models from synthetic aperture sonar big data , 2016, Proceedings of meetings on acoustics.

[5]  James T. Kajiya,et al.  The rendering equation , 1998 .

[6]  David P. Williams.  Underwater target classification in synthetic aperture sonar imagery using deep convolutional neural networks , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[7]  Bolei Zhou,et al.  GAN Dissection: Visualizing and Understanding Generative Adversarial Networks , 2018, ICLR.

[8]  G. S. Sammelmann.  High-frequency images of proud and buried 3D-targets , 2003, Oceans 2003. Celebrating the Past ... Teaming Toward the Future (IEEE Cat. No.03CH37492).

[9]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Shawn Fredrick Johnson.  Synthetic aperture sonar image statistics , 2009 .

[11]  Sen Zhang,et al.  An Interferometric Synthetic Aperture Sonar Raw Signal Simulation Based on Points-Scatterer Model , 2009, 2009 International Joint Conference on Computational Sciences and Optimization.

[12]  Chris Donahue,et al.  Adversarial Audio Synthesis , 2018, ICLR.

[13]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[14]  Richard J. Meyer,et al.  Design considerations for a Compact Correlation Velocity Log , 2018 .

[15]  Alan J. Hunter,et al.  Underwater Acoustic Modelling for Synthetic Aperture Sonar , 2006 .

[16]  G. S. Sammelmann.  Propagation and scattering in very shallow water , 2001, MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295).

[17]  Antti Ilari Karjalainen,et al.  Training and Validation of Automatic Target Recognition Systems using Generative Adversarial Networks , 2019, 2019 Sensor Signal Processing for Defence Conference (SSPD).

[18]  Patrick Pérez,et al.  Sonar image segmentation using an unsupervised hierarchical MRF model , 2000, IEEE Trans. Image Process..

[19]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Daniel C. Brown,et al.  SAS Simulations with Procedural Texture and the Point-based Sonar Scattering Model , 2018, OCEANS 2018 MTS/IEEE Charleston.

[21]  Yves Doisy.  General motion estimation from correlation sonar , 1999 .

[22]  Son-Cheol Yu,et al.  Sonar Image Translation Using Generative Adversarial Network for Underwater Object Recognition , 2019, 2019 IEEE Underwater Technology (UT).

[23]  Sejin Lee,et al.  Deep Learning from Shallow Dives: Sonar Image Generation and Training for Underwater Object Detection , 2018, ArXiv.

[24]  Dajun Tang,et al.  Simulating Realistic-Looking Sediment Ripple Fields , 2009, IEEE Journal of Oceanic Engineering.

[25]  David P. Williams,et al.  On Human Perception and Automatic Target Recognition: Strategies for Human-Computer Cooperation , 2014, 2014 22nd International Conference on Pattern Recognition.

[26]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[27]  A. Bellettini,et al.  Theoretical accuracy of synthetic aperture sonar micronavigation using a displaced phase-center antenna , 2002 .

[28]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[29]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[30]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[31]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Jason R. Stack,et al.  Automation for underwater mine recognition: current trends and future strategy , 2011, Defense + Commercial Sensing.

[33]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[34]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[35]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Ahmad T Abawi,et al.  Kirchhoff scattering from non-penetrable targets modeled as an assembly of triangular facets. , 2016, The Journal of the Acoustical Society of America.

[37]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[38]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[39]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[40]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[41]  S. Reed,et al.  PATT: A performance analysis and training tool for the assessment and adaptive planning of Mine Counter Measure (MCM) operations , 2009, OCEANS 2009.

[42]  David P. Williams.  A Novel Framework for Evaluating Performance-Estimation Models , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[44]  Pascal A. M. de Theije,et al.  Multistatic Sonar Simulations with SIMONA , 2006, 2006 9th International Conference on Information Fusion.

[45]  Daniel C. Brown,et al.  A point-based scattering model for the incoherent component of the scattered field. , 2017, The Journal of the Acoustical Society of America.

[46]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.