A Review of Image Fusion Algorithms Based on the Super-Resolution Paradigm

A critical analysis of remote sensing image fusion methods based on the super-resolution (SR) paradigm is presented in this paper. Very recent algorithms have been selected among the pioneering studies adopting a new methodology and the most promising solutions. After introducing the concept of super-resolution and modeling the approach as a constrained optimization problem, different SR solutions for spatio-temporal fusion and pan-sharpening are reviewed and critically discussed. Concerning pan-sharpening, the well-known, simple, yet effective, proportional additive wavelet in the luminance component (AWLP) is adopted as a benchmark to assess the performance of the new SR-based pan-sharpening methods. The widespread quality indexes computed at degraded resolution, with the original multispectral image used as the reference, i.e., SAM (Spectral Angle Mapper) and ERGAS (Erreur Relative Globale Adimensionnelle de Synthese), are finally presented. Considering these results, sparse representation and Bayesian approaches seem far from being mature to be adopted in operational pan-sharpening scenarios.

[1]  Johannes R. Sveinsson,et al.  A New Pansharpening Algorithm Based on Total Variation , 2014, IEEE Geoscience and Remote Sensing Letters.

[2]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[3]  P. Chavez,et al.  Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis , 1989 .

[4]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Luciano Alparone,et al.  Sensitivity of Pansharpening Methods to Temporal and Instrumental Changes Between Multispectral and Panchromatic Data Sets , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[6]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[7]  Gintautas Palubinskas,et al.  Model-based view at multi-resolution image fusion methods and quality assessment measures , 2016 .

[8]  Xavier Otazu,et al.  Comparison between Mallat's and the ‘à trous’ discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images , 2005 .

[9]  Andrea Garzelli,et al.  Pansharpening of Multispectral Images Based on Nonlocal Parameter Optimization , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Bo Huang,et al.  Spatiotemporal Reflectance Fusion via Sparse Representation , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[12]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[13]  Hankui K. Zhang,et al.  A New Look at Image Fusion Methods from a Bayesian Perspective , 2015, Remote. Sens..

[14]  Yifan Zhang,et al.  A Bayesian Restoration Approach for Hyperspectral Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Henry Leung,et al.  Fusion of Multispectral and Panchromatic Images Using a Restoration-Based Method , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Richard Bamler,et al.  Exploiting Joint Sparsity for Pansharpening: The J-SparseFI Algorithm , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Ye Zhang,et al.  Adaptive Morphological Filtering Method for Structural Fusion Restoration of Hyperspectral Images , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Luciano Alparone,et al.  Spatial Methods for Multispectral Pansharpening: Multiresolution Analysis Demystified , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[20]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[21]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[22]  Te-Ming Tu,et al.  A new look at IHS-like image fusion methods , 2001, Inf. Fusion.

[23]  Luciano Alparone,et al.  A Theoretical Analysis of the Effects of Aliasing and Misregistration on Pansharpened Imagery , 2011, IEEE Journal of Selected Topics in Signal Processing.

[24]  Andrea Garzelli,et al.  PAN‐sharpening of very high resolution multispectral images using genetic algorithms , 2006 .

[25]  Yifan Zhang,et al.  Noise-Resistant Wavelet-Based Bayesian Fusion of Multispectral and Hyperspectral Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[27]  Deepa Kundur,et al.  Blind Image Deconvolution , 2001 .

[28]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[29]  Wenxian Yu,et al.  Sparse representation based pan-sharpening , 2013, 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS.

[30]  Jocelyn Chanussot,et al.  A Pansharpening Method Based on the Sparse Representation of Injected Details , 2015, IEEE Geoscience and Remote Sensing Letters.

[31]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[32]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[33]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Bo Huang,et al.  Spatiotemporal Satellite Image Fusion Through One-Pair Image Learning , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[35]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[36]  Julien Radoux,et al.  Bayesian Data Fusion for Adaptable Image Pansharpening , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Andrea Garzelli,et al.  Panchromatic sharpening of remote sensing images using a multiscale Kalman filter , 2007, Pattern Recognit..

[38]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[39]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[40]  K. Moffett,et al.  Remote Sens , 2015 .

[41]  Robert A. Schowengerdt,et al.  Remote sensing, models, and methods for image processing , 1997 .

[42]  Bruno Aiazzi,et al.  A Comparison Between Global and Context-Adaptive Pansharpening of Multispectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[43]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[45]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[46]  V. K. Shettigara,et al.  A generalized component substitution technique for spatial enhancement of multispectral images using , 1992 .

[47]  Jocelyn Chanussot,et al.  Pansharpening Based on Semiblind Deconvolution , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[49]  Zongben Xu,et al.  Spatial and Spectral Image Fusion Using Sparse Matrix Factorization , 2014, IEEE Transactions on Geoscience and Remote Sensing.