A generative model method for unsupervised multispectral image fusion in remote sensing

This paper presents a generative model method for multispectral image fusion in remote sensing which involves training without supervision. This method eases the supervision of learning and also uses a multi-objective loss function to achieve image fusion. The loss function used incorporates both spectral and spatial distortions. Two discriminators are designed to minimize the spectral and spatial distortions of the generative output. Extensive experimentations are conducted using three public domain datasets. The comparison results across four reduced-resolution and three full-resolution objective metrics show the superiority of the developed method over several recently developed methods.

[1]  Munchurl Kim,et al.  Pan-Sharpening With Color-Aware Perceptual Loss And Guided Re-Colorization , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[2]  Hassan Ghassemian,et al.  A new pansharpening method using multi resolution analysis framework and deep neural networks , 2017, 2017 3rd International Conference on Pattern Recognition and Image Analysis (IPRIA).

[3]  Ying Li,et al.  Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening , 2019, Remote. Sens..

[4]  Jae-Seok Choi,et al.  S3: A Spectral-Spatial Structure Loss for Pan-Sharpening Networks , 2020, IEEE Geoscience and Remote Sensing Letters.

[5]  Shuyuan Yang,et al.  Novel Adaptive Component-Substitution-Based Pan-Sharpening Using Particle Swarm Optimization , 2015, IEEE Geoscience and Remote Sensing Letters.

[6]  Xueliang Zhang,et al.  Deep learning in remote sensing applications: A meta-analysis and review , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[7]  Gaurav Sharma,et al.  A Regularized Model-Based Optimization Framework for Pan-Sharpening , 2014, IEEE Transactions on Image Processing.

[8]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Jon Atli Benediktsson,et al.  Pansharpening With Matting Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Jocelyn Chanussot,et al.  Multi-resolution analysis techniques and nonlinear PCA for hybrid pansharpening applications , 2016, Multidimens. Syst. Signal Process..

[11]  Hassan Ghassemian,et al.  Application of fractional-order differentiation in multispectral image fusion , 2018 .

[12]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[14]  Nasser Kehtarnavaz,et al.  Convolutional Autoencoder-Based Multispectral Image Fusion , 2019, IEEE Access.

[15]  Liangpei Zhang,et al.  A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Hassan Ghassemian,et al.  An adaptive multispectral image fusion using particle swarm optimization , 2017, 2017 Iranian Conference on Electrical Engineering (ICEE).

[17]  Sergio Vitale.  A CNN-Based Pansharpening Method with Perceptual Loss , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[18]  Qing Liu,et al.  A regularised model-based pan-sharpening method for remote sensing images with local dissimilarities , 2018, International Journal of Remote Sensing.

[19]  Hongyi Liu,et al.  A New Pan-Sharpening Method With Deep Neural Networks , 2015, IEEE Geoscience and Remote Sensing Letters.

[20]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[22]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Giuseppe Scarpa,et al.  A Detail-Preserving Cross-Scale Learning Strategy for CNN-Based Pansharpening , 2020, Remote. Sens..

[24]  Xiangyu Liu,et al.  Psgan: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[25]  Nikolaos Mitianoudis,et al.  Fast wavelet-based pansharpening of multi-spectral images , 2010, 2010 IEEE International Conference on Imaging Systems and Techniques.

[26]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Jie Liang,et al.  Saliency Analysis and Gaussian Mixture Model-Based Detail Extraction Algorithm for Hyperspectral Pansharpening , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Ye Zhang,et al.  Combining Component Substitution and Multiresolution Analysis: A Novel Generalized BDSD Pansharpening Algorithm , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[29]  Nasser Kehtarnavaz,et al.  Multispectral image fusion based on map estimation with improved detail , 2020 .

[30]  Nasser Kehtarnavaz,et al.  Image fusion in remote sensing by multi-objective deep learning , 2020 .

[31]  Jocelyn Chanussot,et al.  A critical comparison of pansharpening algorithms , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[32]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[33]  Liangpei Zhang,et al.  Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[34]  Davide Cozzolino,et al.  Target-Adaptive CNN-Based Pansharpening , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[36]  Junfeng Yang,et al.  PanNet: A Deep Network Architecture for Pan-Sharpening , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Johannes R. Sveinsson,et al.  Optimal Component Substitution and Multi-Resolution Analysis Pansharpening Methods Using a Convolutional Neural Network , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.