Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics.

[1]  Manjunath V. Joshi,et al.  Multiresolution Image Fusion: Use of Compressive Sensing and Graph Cuts , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  V. K. Shettigara,et al.  A generalized component substitution technique for spatial enhancement of multispectral images using , 1992 .

[3]  Truong Q. Nguyen,et al.  Wavelets and filter banks , 1996 .

[4]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[5]  Lorenzo Bruzzone,et al.  Can multiresolution fusion techniques improve classification accuracy? , 2006, SPIE Remote Sensing.

[6]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[7]  Lucien Wald,et al.  A Method to Better Account for Modulation Transfer Functions in ARSIS-Based Pansharpening Methods , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[8]  L. Alparone,et al.  An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas , 2003, 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas.

[9]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[10]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[11]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[12]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[13]  J. Schott,et al.  Resolution enhancement of multispectral image data to improve classification accuracy , 1993 .

[14]  P. Chavez,et al.  Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis , 1989 .

[15]  Jiangshe Zhang,et al.  An Improved Adaptive Intensity–Hue–Saturation Method for the Fusion of Remote Sensing Images , 2014, IEEE Geoscience and Remote Sensing Letters.

[16]  Te-Ming Tu,et al.  A new look at IHS-like image fusion methods , 2001, Inf. Fusion.

[17]  Javier Marcello,et al.  Quality evaluation of pansharpening techniques on different land cover types , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[18]  Johannes R. Sveinsson,et al.  Model-Based Fusion of Multi- and Hyperspectral Images Using PCA and Wavelets , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Jon Atli Benediktsson,et al.  Pansharpening With Matting Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Johannes R. Sveinsson,et al.  Classification of Pansharpened Urban Satellite Images , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[21]  Johannes R. Sveinsson,et al.  A New Pansharpening Algorithm Based on Total Variation , 2014, IEEE Geoscience and Remote Sensing Letters.

[22]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[23]  Lucien Wald,et al.  Comparing distances for quality assessment of fused images , 2006 .

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  Jocelyn Chanussot,et al.  Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique , 2008, IEEE Geoscience and Remote Sensing Letters.

[26]  Guy Flouzat,et al.  Thematic and statistical evaluations of five panchromatic/multispectral fusion methods on simulated PLEIADES-HR images , 2005, Inf. Fusion.

[27]  Jocelyn Chanussot,et al.  A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors , 2014, IEEE Transactions on Image Processing.

[28]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Te-Ming Tu,et al.  A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[30]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Liangpei Zhang,et al.  Quality Assessment of Panchromatic and Multispectral Image Fusion for the ZY-3 Satellite: From an Information Extraction Perspective , 2014, IEEE Geoscience and Remote Sensing Letters.

[32]  Gintautas Palubinskas.  Quality assessment of pan-sharpening methods , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[33]  L. Wald,et al.  Fusion of high spatial and spectral resolution images : The ARSIS concept and its implementation , 2000 .

[34]  Bruno Aiazzi,et al.  Influence of spatial resolution on pan-sharpening results , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[35]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[36]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[38]  Nicolas H. Younan,et al.  Pansharpening and image quality interface , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.

[39]  Robert A. Schowengerdt,et al.  Remote sensing, models, and methods for image processing , 1997 .

[40]  Qian Du,et al.  Performance evaluation for pan-sharpening techniques , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[41]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[42]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[43]  Lucien Wald,et al.  Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions , 2002 .

[44]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[45]  I. Pippi,et al.  Quality assessment of decision-driven pyramid-based fusion of high resolution multispectral with panchromatic image data , 2001, IEEE/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas (Cat. No.01EX482).

[46]  Danilo P. Mandic,et al.  A Multivariate Empirical Mode DecompositionBased Approach to Pansharpening , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[47]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Consuelo Gonzalo-Martín,et al.  Evaluation of the performance of spatial assessments of pansharpened images , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[49]  Alan R. Gillespie,et al.  Color enhancement of highly correlated images. II. Channel ratio and “chromaticity” transformation techniques , 1987 .

[50]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[51]  P. Dutilleux.  An Implementation of the “algorithme à trous” to Compute the Wavelet Transform , 1989 .

[52]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[53]  Jocelyn Chanussot,et al.  Pansharpening Quality Assessment Using the Modulation Transfer Functions of Instruments , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[54]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[55]  Guian Wang,et al.  Performance evaluation of different references based image fusion quality metrics for quality assessment of remote sensing Image fusion , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.