Model-based variational pansharpening method with fast generalized intensity–hue–saturation

Abstract. Pansharpening is a process used to extract high spatial resolution from panchromatic images and high spectral resolution from multispectral images at the same time. The intensity–hue–saturation (IHS) is the simplest and most efficient fusion method in current fusion methods, which could preserve spatial information well, but there is some spectral distortion. The main reason is that, in the IHS space, the estimated intensity image used to replace the original intensity image is not accurate enough. A method is proposed to overcome this defect, which aims to combine the fast generalized IHS (FGIHS) fusion technique based on spectral adjustment with a variational optimization model to estimate a more accurate intensity image. In the variational model, it is assumed that the down-sampled intensity image should be close to the original intensity image, retaining the spectral information effectively. The experimental results show that this method not only preserves the spectral information effectively but also improves the spatial quality of the fusion results in the view of the subjective and objective evaluations.

[1]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[2]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[3]  Hankui K. Zhang,et al.  A New Look at Image Fusion Methods from a Bayesian Perspective , 2015, Remote. Sens..

[4]  Haixu Wang,et al.  Multimodal medical image fusion based on IHS and PCA , 2010 .

[5]  Te-Ming Tu,et al.  A new look at IHS-like image fusion methods , 2001, Inf. Fusion.

[6]  Gulcan Sarp,et al.  Spectral and spatial quality analysis of pan-sharpening algorithms: A case study in Istanbul , 2014 .

[7]  H. Abdi,et al.  Principal component analysis , 2010 .

[8]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[9]  Fang Li,et al.  A Variational Approach for Pan-Sharpening , 2013, IEEE Transactions on Image Processing.

[10]  Bin Xiao,et al.  Adaptive remote-sensing image fusion based on dynamic gradient sparse and average gradient difference , 2017 .

[11]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[12]  S. Ashraf,et al.  Image data fusion for the remote sensing of freshwater environments , 2012 .

[13]  Veronika Kopacková,et al.  Testing a Modified PCA-Based Sharpening Approach for Image Fusion , 2016, Remote. Sens..

[14]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[15]  Wei Liu,et al.  SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified Framework , 2015, IEEE Transactions on Image Processing.

[16]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Laura Igual,et al.  A Variational Model for P+XS Image Fusion , 2006, International Journal of Computer Vision.

[19]  Thomas Maurer HOW TO PAN-SHARPEN IMAGES USING THE GRAM-SCHMIDT PAN-SHARPEN METHOD – A RECIPE , 2013 .

[20]  Lorenzo Bruzzone,et al.  Image fusion techniques for remote sensing applications , 2002, Inf. Fusion.

[21]  Jiangshe Zhang,et al.  An Improved Adaptive Intensity–Hue–Saturation Method for the Fusion of Remote Sensing Images , 2014, IEEE Geoscience and Remote Sensing Letters.

[22]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[23]  Mark W. Schmidt,et al.  Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization , 2011, NIPS.

[24]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[25]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[26]  Zeming Zhou,et al.  Model-based variational fusion for reducing spectral distortion , 2016, Science China Information Sciences.

[27]  Qingming Zhan,et al.  An IHS-Based Pan-Sharpening Method for Spectral Fidelity Improvement Using Ripplet Transform and Compressed Sensing , 2018, Sensors.

[28]  Jocelyn Chanussot,et al.  Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique , 2008, IEEE Geoscience and Remote Sensing Letters.

[29]  Andrea Garzelli,et al.  A Review of Image Fusion Algorithms Based on the Super-Resolution Paradigm , 2016, Remote. Sens..

[30]  Shuyuan Yang,et al.  Deep Sparse Tensor Filtering Network for Synthetic Aperture Radar Images Classification , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[32]  Multispectral and Hyperspectral Pansharpening : A Critical Examination and New Developments , 2014 .

[33]  Wenzhong Shi,et al.  Multisource Image Fusion Method Using Support Value Transform , 2007, IEEE Transactions on Image Processing.

[34]  Xiao Xiang Zhu,et al.  Compressive Sensing for PAN-Sharpening , 2011 .

[35]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Jinying Zhong,et al.  Remote Sensing Image Fusion with Convolutional Neural Network , 2016 .

[37]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[38]  John van Genderen,et al.  Structuring contemporary remote sensing image fusion , 2015 .

[39]  David P. Roy,et al.  Computationally Inexpensive Landsat 8 Operational Land Imager (OLI) Pansharpening , 2016, Remote. Sens..

[40]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[41]  Hassan Ghassemian,et al.  Nonlinear IHS: A Promising Method for Pan-Sharpening , 2016, IEEE Geoscience and Remote Sensing Letters.

[42]  Shuyuan Yang,et al.  Learning Low-Rank Decomposition for Pan-Sharpening With Spatial-Spectral Offsets , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[44]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[45]  Michael Möller,et al.  A Variational Approach for Sharpening High Dimensional Images , 2012, SIAM J. Imaging Sci..