Quality Assessment of SAR-to-Optical Image Translation

Synthetic aperture radar (SAR) images contain severe speckle noise and weak texture, which are unsuitable for visual interpretation. Many studies have been undertaken so far toward exploring the use of SAR-to-optical image translation to obtain near optical representations. However, how to evaluate the translation quality is a challenge. In this paper, we combine image quality assessment (IQA) with SAR-to-optical image translation to pursue a suitable evaluation approach. Firstly, several machine-learning baselines for SAR-to-optical image translation are established and evaluated. Then, extensive comparisons of perceptual IQA models are performed in terms of their use as objective functions for the optimization of image restoration. In order to study feature extraction of the images translated from SAR to optical modes, an application in scene classification is presented. Finally, the attributes of the translated image representations are evaluated using visual inspection and the proposed IQA methods.

[1]  Shiqi Wang,et al.  Image Quality Assessment: Unifying Structure and Texture Similarity , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Shiqi Wang,et al.  Comparison of Image Quality Models for Optimization of Image Processing Systems , 2020, ArXiv.

[3]  Jianjiang Zhou,et al.  Feature-Guided SAR-to-Optical Image Translation , 2020, IEEE Access.

[4]  Yue Yu,et al.  SAR-to-Optical Image Translation Using Supervised Cycle-Consistent Adversarial Networks , 2019, IEEE Access.

[5]  Corentin Henry,et al.  SAR-to-Optical Image Translation Based on Conditional Generative Adversarial Networks - Optimization, Opportunities and Limits , 2019, Remote. Sens..

[6]  Itaru Kitahara,et al.  SAR2OPT: Image Alignment Between Multi-Modal Images Using Generative Adversarial Networks , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[7]  Zhiqiang Shen,et al.  Towards Instance-Level Image-To-Image Translation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ling Shao,et al.  iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images , 2019, CVPR Workshops.

[9]  Feng Xu,et al.  Translating SAR to Optical Images for Assisted Interpretation , 2019, ArXiv.

[10]  Xiao Xiang Zhu,et al.  The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion , 2018, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[11]  Weiming Wang,et al.  Image Translation Between Sar and Optical Imagery with Generative Adversarial Nets , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[12]  Olaf Hellwich,et al.  Exploiting GAN-Based SAR to Optical Image Transcoding for Improved Classification via Deep Learning , 2018 .

[13]  Jing Huang,et al.  DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[19]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Renjie Liao,et al.  Learning to generate images with perceptual similarity metrics , 2015, 2017 IEEE International Conference on Image Processing (ICIP).

[22]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[23]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[27]  K. Moffett,et al.  Remote Sens , 2015 .

[28]  Luciano Alparone,et al.  A Tutorial on Speckle Reduction in Synthetic Aperture Radar Images , 2013, IEEE Geoscience and Remote Sensing Magazine.

[29]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[30]  Wen Gao,et al.  SSIM-Motivated Rate-Distortion Optimization for Video Coding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Zhou Wang,et al.  Applications of Objective Image Quality Assessment Methods [Applications Corner] , 2011, IEEE Signal Processing Magazine.

[32]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[33]  Dengxin Dai,et al.  Satellite Image Classification via Two-Layer Sparse Coding With Biased Image Representation , 2011, IEEE Geoscience and Remote Sensing Letters.

[34]  Zhou Wang,et al.  Applications of Objective Image Quality Assessment Methods , 2011 .

[35]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[36]  Robert W. Heath,et al.  SSIM-optimal linear image restoration , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[38]  Arthur B. Markman,et al.  Nonintentional similarity processing , 2005 .

[39]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[40]  Neil W. Bergmann,et al.  An automatic image quality assessment technique incorporating higher level perceptual factors , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).