Pansharpening Based on Low-Rank Fuzzy Fusion and Detail Supplement

Pansharpening is a technique used to reconstruct a high-resolution (HR) multispectral (MS) image by combining an HR panchromatic (PAN) image with a low-resolution MS image. In recent years, the detail-injection model has demonstrated excellent performance in pansharpening, thus receiving wide attention. Obtaining appropriate details is vital for the detail-injection model. Therefore, this article presents a detail optimization approach to obtain more precise high-frequency (HF) details for pansharpening. The proposed method comprises two steps. In the first step, we design a low-rank fuzzy fusion model to fuse the HF details of the PAN and MS images. In this model, the high frequencies of the PAN and upsampled MS images are decomposed into low-rank and sparse components, and the corresponding fusion rules are designed according to their characteristics. Because some details of the PAN image are replaced with those of the MS image, using them directly as injection details may result in redundant information or spatial distortion. To solve this problem and further optimize the details, in the second step, we construct an adaptive detail supplement model. Based on the similarity and correlation between the fused HF and the original HF of the PAN image, the fused details are supplemented to obtain the final injection details. Experimental results on the IKONOS, Pleiades, QuickBird, and WorldView-2 datasets demonstrate that the proposed algorithm is better than the state-of-the-art methods in maintaining spectral information and improving spatial details.

[1]  Yaonan Wang,et al.  Using the discrete wavelet frame transform to merge Landsat TM and SPOT panchromatic images , 2002, Inf. Fusion.

[2]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[3]  Rob Heylen,et al.  Fusion of Hyperspectral and Multispectral Images Using Spectral Unmixing and Sparse Coding , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  V. K. Shettigara,et al.  A generalized component substitution technique for spatial enhancement of multispectral images using , 1992 .

[5]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[6]  Lorenzo Bruzzone,et al.  Image fusion techniques for remote sensing applications , 2002, Inf. Fusion.

[7]  José A. Malpica,et al.  Hue Adjustment to IHS Pan-Sharpened IKONOS Imagery for Vegetation Enhancement , 2007, IEEE Geoscience and Remote Sensing Letters.

[8]  Pan Lin,et al.  A Novel Pan-Sharpening Framework Based on Matting Model and Multiscale Transform , 2017, Remote. Sens..

[9]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[10]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[11]  Wei Liu,et al.  SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified Framework , 2015, IEEE Transactions on Image Processing.

[12]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[13]  Michael Möller,et al.  A Variational Approach for Sharpening High Dimensional Images , 2012, SIAM J. Imaging Sci..

[14]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[15]  S. Baronti,et al.  Sharpening of very high resolution images with spectral distortion minimization , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[16]  Liangpei Zhang,et al.  A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Luciano Alparone,et al.  Sensitivity of Pansharpening Methods to Temporal and Instrumental Changes Between Multispectral and Panchromatic Data Sets , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Michael Möller,et al.  An Adaptive IHS Pan-Sharpening Method , 2010, IEEE Geoscience and Remote Sensing Letters.

[20]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[21]  Mario Chica-Olmo,et al.  Downscaling cokriging for image sharpening , 2006 .

[22]  Song Xiao,et al.  An Improved Hyperspectral Pansharpening Algorithm Based on Optimized Injection Model , 2019, IEEE Access.

[23]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[24]  Qiuming Cheng,et al.  An image fusion method for misaligned panchromatic and multispectral data , 2011 .

[25]  Isin Erer,et al.  Bilateral Filtering-Based Enhanced Pansharpening of Multispectral Satellite Images , 2014, IEEE Geoscience and Remote Sensing Letters.

[26]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[27]  Altan Mesut,et al.  A comparative analysis of image fusion methods , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[28]  Jean-Yves Tourneret,et al.  Bayesian Fusion of Multi-Band Images , 2013, IEEE Journal of Selected Topics in Signal Processing.

[29]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[30]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[31]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[32]  Xiaoqun Zhang,et al.  Pansharpening Image Fusion Using Cross-Channel Correlation: A Framelet-Based Approach , 2015, Journal of Mathematical Imaging and Vision.

[33]  J. Chassery,et al.  The use of multiresolution analysis and wavelets transform for merging SPOT panchromatic and multisp , 1996 .

[34]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Wei Gao,et al.  Multisensor Satellite Image Fusion and Networking for All-Weather Environmental Monitoring , 2018, IEEE Systems Journal.

[36]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[37]  Laura Igual,et al.  A Variational Model for P+XS Image Fusion , 2006, International Journal of Computer Vision.

[38]  Liangpei Zhang,et al.  An Integrated Framework for the Spatio–Temporal–Spectral Fusion of Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Wenzhong Shi,et al.  Information Loss-Guided Multi-Resolution Image Fusion , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Jon Atli Benediktsson,et al.  Pansharpening With Matting Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Feiniu Yuan,et al.  Remote Sensing Image Fusion Based on Adaptive IHS and Multiscale Guided Filter , 2016, IEEE Access.

[43]  Fang Liu,et al.  Coupled Tensor Decomposition for Hyperspectral Pansharpening , 2018, IEEE Access.

[44]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[45]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Hankui K. Zhang,et al.  A New Look at Image Fusion Methods from a Bayesian Perspective , 2015, Remote. Sens..

[47]  Bruno Aiazzi,et al.  A Comparison Between Global and Context-Adaptive Pansharpening of Multispectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[48]  Yu Guo,et al.  A novel image fusion algorithm based on nonsubsampled shearlet transform , 2014 .

[49]  Jinze Yu,et al.  Rank-constrained PCA for intrinsic images decomposition , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[50]  Jocelyn Chanussot,et al.  Full Scale Regression-Based Injection Coefficients for Panchromatic Sharpening , 2018, IEEE Transactions on Image Processing.

[51]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Pan Lin,et al.  Multimodal Sensor Medical Image Fusion Based on Type-2 Fuzzy Logic in NSCT Domain , 2016, IEEE Sensors Journal.

[53]  Pan Lin,et al.  Multifocus Image Fusion Based on NSCT and Focused Area Detection , 2014, IEEE Sensors Journal.

[54]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[55]  Jeffrey A. Fessler,et al.  Improved Robust PCA using low-rank denoising with optimal singular value shrinkage , 2014, 2014 IEEE Workshop on Statistical Signal Processing (SSP).

[56]  Linhai Jing,et al.  Improvement of a Pansharpening Method Taking Into Account Haze , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[57]  Hassan Ghassemian,et al.  A review of remote sensing image fusion methods , 2016, Inf. Fusion.

[58]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[59]  Robert A. Schowengerdt,et al.  Reconstruction of multispatial, multispectral image data using spatial frequency content , 1980 .

[60]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[61]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[62]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[63]  Lei Wu,et al.  Remote Sensing Image Fusion Based on Adaptively Weighted Joint Detail Injection , 2018, IEEE Access.

[64]  Liangpei Zhang,et al.  Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[65]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.