Dialectical GAN for SAR Image Translation: From Sentinel-1 to TerraSAR-X

Contrary to optical images, Synthetic Aperture Radar (SAR) images are in different electromagnetic spectrum where the human visual system is not accustomed to. Thus, with more and more SAR applications, the demand for enhanced high-quality SAR images has increased considerably. However, high-quality SAR images entail high costs due to the limitations of current SAR devices and their image processing resources. To improve the quality of SAR images and to reduce the costs of their generation, we propose a Dialectical Generative Adversarial Network (Dialectical GAN) to generate high-quality SAR images. This method is based on the analysis of hierarchical SAR information and the "dialectical" structure of GAN frameworks. As a demonstration, a typical example will be shown where a low-resolution SAR image (e.g., a Sentinel-1 image) with large ground coverage is translated into a high-resolution SAR image (e.g., a TerraSAR-X image). Three traditional algorithms are compared, and a new algorithm is proposed based on a network framework by combining conditional WGAN-GP (Wasserstein Generative Adversarial Network - Gradient Penalty) loss functions and Spatial Gram matrices under the rule of dialectics. Experimental results show that the SAR image translation works very well when we compare the results of our proposed method with the selected traditional methods.

[1]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Peter Reinartz,et al.  Exploring the Potential of Conditional Adversarial Networks for Optical and SAR Image Matching , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Tao Tang,et al.  A Kernel Clustering Algorithm With Fuzzy Factor: Application to SAR Image Segmentation , 2014, IEEE Geoscience and Remote Sensing Letters.

[4]  Simon Hessner,et al.  Image Style Transfer using Convolutional Neural Networks , 2018 .

[5]  Abhinav Gupta,et al.  Generative Image Modeling Using Style and Structure Adversarial Networks , 2016, ECCV.

[6]  Mario Lucic,et al.  Are GANs Created Equal? A Large-Scale Study , 2017, NeurIPS.

[7]  J. Marchand,et al.  SAR Image Quality Assessment , 1993 .

[8]  Zunlei Feng,et al.  Neural Style Transfer: A Review , 2017, IEEE Transactions on Visualization and Computer Graphics.

[9]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[10]  Torbjørn Eltoft,et al.  Estimation of the Equivalent Number of Looks in Polarimetric Synthetic Aperture Radar Imagery , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[12]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Xiao Xiang Zhu,et al.  Deep learning in remote sensing: a review , 2017, ArXiv.

[14]  Alexei A. Efros,et al.  Generative Visual Manipulation on the Natural Image Manifold , 2016, ECCV.

[15]  Davide Cozzolino,et al.  SAR image despeckling through convolutional neural networks , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[16]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  Cheng Hu,et al.  Performance and Requirements of GEO SAR Systems in the Presence of Radio Frequency Interferences , 2018, Remote. Sens..

[19]  Xiaojun Yang,et al.  Spatial Interpolation , 2017, Encyclopedia of GIS.

[20]  Xiaofeng Li,et al.  Using SAR images to delineate ocean oil slicks with a texture-classifying neural network algorithm (TCNNA) , 2009 .

[21]  Dmitry Yarotsky,et al.  Optimal approximation of continuous functions by very deep ReLU networks , 2018, COLT.

[22]  Cheng Hu,et al.  A Sparse SAR Imaging Method Based on Multiple Measurement Vectors Model , 2017, Remote. Sens..

[23]  Geoffrey E. Hinton Reducing the Dimensionality of Data with Neural , 2008 .

[24]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  René Vidal,et al.  Global Optimality in Neural Network Training , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Cheng Hu,et al.  Optimal 3D deformation measuring in inclined geosynchronous orbit SAR differential interferometry , 2016, Science China Information Sciences.

[27]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Gang Hua,et al.  Visual attribute transfer through deep image analogy , 2017, ACM Trans. Graph..

[29]  Andrea Vedaldi,et al.  Texture Networks: Feed-forward Synthesis of Textures and Stylized Images , 2016, ICML.

[30]  Jeffrey Pennington,et al.  Geometry of Neural Network Loss Surfaces via Random Matrix Theory , 2017, ICML.

[31]  Mihai Datcu,et al.  Land Cover Semantic Annotation Derived from High-Resolution SAR Images , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[32]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[33]  Jian Yang,et al.  A novel method of speckle reduction and enhancement for SAR image , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[34]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Roland Memisevic,et al.  Incorporating long-range consistency in CNN-based texture generation , 2016, ICLR.

[36]  Weiming Tian,et al.  Accurate Analysis of Target Characteristic in Bistatic SAR Images: A Dihedral Corner Reflectors Case , 2017, Sensors.

[37]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[38]  Pavel Zezula,et al.  Metric Space , 2009, Encyclopedia of Database Systems.

[39]  R. Srikant,et al.  Why Deep Neural Networks for Function Approximation? , 2016, ICLR.

[40]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.