Cartoon-Texture Decomposition-Based Variational Pansharpening

Pansharpening is widely used to increase the spatial resolution of a multispectral (MS) image by fusing with a panchromatic (PAN) image that has high-spatial resolution and the same scene. In this paper, the similarities of MS and PAN images in cartoon-texture space are exploited. The cartoon and texture components of an image always contain the global structure information and the locally-patterned information, respectively. Therefore, the global and local spatial details (i.e., high-order information) could be preserved well in the fused high-spatial resolution MS image after leveraging the similarities of these images. Pansharpening is formulated as the optimization problem with respect to the cartoon-texture similarities between the MS and the PAN images in this work based on the aforementioned observation. Specifically, cartoon similarity is determined through gradient sparsity and formulated as a total variation term, whereas texture similarity is described according to the low-rank property. The alternative direction multiplier method is used to solve the optimization problem. In the experiment, the Gaofen-1 satellite dataset is used to compare the proposed method with other classical pansharpening methods. Experimental results demonstrate that our method outperforms the comparison methods in terms of visual and quantitative qualities.

[1]  Yonghyun Kim,et al.  Improved Additive-Wavelet Image Fusion , 2011, IEEE Geoscience and Remote Sensing Letters.

[2]  Jocelyn Chanussot,et al.  Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique , 2008, IEEE Geoscience and Remote Sensing Letters.

[3]  Shunsuke Ono,et al.  Cartoon-Texture Image Decomposition Using Blockwise Low-Rank Texture Characterization , 2014, IEEE Transactions on Image Processing.

[4]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[5]  Laura Igual,et al.  A Variational Model for P+XS Image Fusion , 2006, International Journal of Computer Vision.

[6]  T. Chan,et al.  Fast dual minimization of the vectorial total variation norm and applications to color image processing , 2008 .

[7]  Fang Li,et al.  A Variational Approach for Pan-Sharpening , 2013, IEEE Transactions on Image Processing.

[8]  Wei Liu,et al.  SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified Framework , 2015, IEEE Transactions on Image Processing.

[9]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[11]  Aggelos K. Katsaggelos,et al.  A survey of classical methods and new trends in pansharpening of multispectral images , 2011, EURASIP J. Adv. Signal Process..

[12]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[13]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[14]  Xinghao Ding,et al.  A Variational Pan-Sharpening With Local Gradient Constraints , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jon Atli Benediktsson,et al.  Pansharpening With Matting Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[16]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[17]  Altan Mesut,et al.  A comparative analysis of image fusion methods , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[18]  Junjun Jiang,et al.  FusionGAN: A generative adversarial network for infrared and visible image fusion , 2019, Inf. Fusion.

[19]  Michael Möller,et al.  An Adaptive IHS Pan-Sharpening Method , 2010, IEEE Geoscience and Remote Sensing Letters.

[20]  Hassan Ghassemian,et al.  Nonlinear IHS: A Promising Method for Pan-Sharpening , 2016, IEEE Geoscience and Remote Sensing Letters.

[21]  Johannes R. Sveinsson,et al.  MTF-Based Deblurring Using a Wiener Filter for CS and MRA Pansharpening Methods , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[22]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[23]  Bruno Aiazzi,et al.  A Comparison Between Global and Context-Adaptive Pansharpening of Multispectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[24]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[25]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .