Object-based spectral quality assessment of high-resolution pan-sharpened satellite imageries: new combined fusion strategy to increase the spectral quality

Panchromatic and multispectral images produced by the earth observation satellites are fused; hence, a high-resolution multispectral image is obtained. Spectral quality of the fused images is of great importance since the quality of a large number of remote sensing products mainly depends on this feature. Due to the importance of the spectral quality of the fused images, its assessment is of particular significance as well. This article proposes an object-based strategy for the spectral quality assessment of the fused images to eliminate the limitations of the current pixel-based method. This kind of assessment is performed by focusing on homogeneous objects with similar spectral and textural behaviors. After determining an optimal metric, the object-based scheme was applied to five datasets from four types of satellite sensors and the spectral behavior of fusion methods was examined within the image classes. Although the spectral behavior of the fusion methods was not regular, the best methods in each class were determined using statistical analysis. Furthermore, a scheme was proposed to combine the results of different fusion methods to obtain a fused image with the best possible spectral quality. The obtained results indicate that this image enjoys 37% better quality than the best-fused image selected based on the pixel-based quality assessment.

[1]  Yun Zhang,et al.  METHODS FOR IMAGE FUSION QUALITY ASSESSMENT - A REVIEW, COMPARISON AND ANALYSIS , 2008 .

[2]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[3]  Hassan Ghassemian,et al.  Nonlinear IHS: A Promising Method for Pan-Sharpening , 2016, IEEE Geoscience and Remote Sensing Letters.

[4]  Yue Huang,et al.  Pan-sharpening with structural consistency and ℓ1/2 gradient prior , 2016 .

[5]  Yonghyun Kim,et al.  Improved Additive-Wavelet Image Fusion , 2011, IEEE Geoscience and Remote Sensing Letters.

[6]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[7]  Yun Zhang,et al.  Wavelet based image fusion techniques — An introduction, review and comparison , 2007 .

[8]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[9]  R. J. Bhiwani,et al.  Image Fusion in Remote Sensing Applications: A Review , 2015 .

[10]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[11]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[12]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Thierry Ranchin,et al.  Liu 'Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details' , 2002 .

[14]  F. Samadzadegan,et al.  New Object Level Strategy for Image Fusion Quality Assessment of High Resolution Satellite Imagery , 2011 .

[15]  Liviu Florin Zoran,et al.  QUALITY EVALUATION OF MULTIRESOLUTION REMOTE SENSING IMAGES FUSION , 2009 .

[16]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Xavier Otazu,et al.  Comparison between Mallat's and the ‘à trous’ discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images , 2005 .

[18]  Jon Atli Benediktsson,et al.  Pansharpening With Matting Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Gwendoline Blanchet,et al.  A Survey of Pansharpening Methods with A New Band-Decoupled Variational Model , 2016, ArXiv.

[20]  T. Ranchin,et al.  Liu 'Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details' , 2002 .

[21]  F. Samadzadegan,et al.  An object-level strategy for pan-sharpening quality assessment of high-resolution satellite imagery , 2014 .

[22]  Laércio Massaru Namikawa,et al.  Image Fusion for Remote Sensing Applications , 2011 .

[23]  W. Shi,et al.  Wavelet-based image fusion and quality assessment , 2005 .

[24]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[25]  L. Alparone,et al.  An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas , 2003, 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas.

[26]  Thomas Seidl,et al.  k-Nearest Neighbor Classification , 2009, Encyclopedia of Database Systems.

[27]  Farhad Samadzadegan,et al.  Spatial Quality Assessment of Pan-Sharpened High Resolution Satellite Imagery Based on an Automatically Estimated Edge Based Metric , 2013, Remote. Sens..

[29]  Wen Dou,et al.  Image Degradation for Quality Assessment of Pan-Sharpening Methods , 2018, Remote. Sens..

[30]  W. Tobler A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[31]  Henk J. A. M. Heijmans,et al.  A new quality metric for image fusion , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[32]  M. Binard,et al.  Fusion of multispectral and panchromatic images by local mean and variance matching filtering techniques , 1998 .

[33]  Nicolas H. Younan,et al.  Quantitative analysis of pansharpened images , 2006 .

[34]  C. Padwick,et al.  WORLDVIEW-2 PAN-SHARPENING , 2010 .

[35]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[36]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Snehmani,et al.  A comparative analysis of pansharpening techniques on QuickBird and WorldView-3 images , 2017 .

[38]  Yun Zhang,et al.  An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images , 2005, Inf. Fusion.

[39]  Genshe Chen,et al.  Image quality assessment for performance evaluation of image fusion , 2008, 2008 11th International Conference on Information Fusion.

[40]  Myungjin Choi,et al.  A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter , 2006, IEEE Trans. Geosci. Remote. Sens..

[41]  Farhad Samadzadegan,et al.  Evaluating the Sensitivity of Image Fusion Quality Metrics to Image Degradation in Satellite Imagery , 2011 .

[42]  U. Benz,et al.  Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information , 2004 .

[43]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Lucien Wald,et al.  Analysis of Changes in Quality Assessment with Scale , 2006, 2006 9th International Conference on Information Fusion.

[45]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[46]  A. Hegde,et al.  A Review of Quality Metrics for Fused Image , 2015 .

[47]  Timo Rolf Bretschneider,et al.  Objective content-dependent quality measures for image fusion of optical data , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.

[48]  Francisco Eugenio,et al.  Object-based quality evaluation procedure for fused remote sensing imagery , 2017, Neurocomputing.

[49]  Pragati Upadhyay,et al.  PIXEL-LEVEL IMAGE FUSION USINGBROVEY TRANSFORME AND WAVELETTRANSFORM , 2013 .

[50]  Luciano Alparone,et al.  Pan-sharpening of multispectral images: a critical review and comparison , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.

[51]  Multiscale contrast image fusion scheme with performance measures , 2004 .

[52]  R.S. Blum,et al.  Experimental tests of image fusion for night vision , 2005, 2005 7th International Conference on Information Fusion.

[53]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[54]  Mahdi Hasanlou,et al.  Quality assessment of pan-sharpening methods in high-resolution satellite images using radiometric and geometric index , 2015, Arabian Journal of Geosciences.

[55]  Xavier Otazu,et al.  Multiresolution-based image fusion with additive wavelet decomposition , 1999, IEEE Trans. Geosci. Remote. Sens..

[56]  Gemine Vivone,et al.  Perceptual Quality Assessment of Pan-Sharpened Images , 2019, Remote. Sens..

[57]  Hassan Ghassemian,et al.  A review of remote sensing image fusion methods , 2016, Inf. Fusion.

[58]  L. Wald,et al.  Fusion of high spatial and spectral resolution images : The ARSIS concept and its implementation , 2000 .

[59]  Rafael García,et al.  Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Josaphat Tetuko Sri Sumantyo,et al.  Assessment of pan-sharpening methods applied to image fusion of remotely sensed multi-band data , 2012, Int. J. Appl. Earth Obs. Geoinformation.

[61]  Jocelyn Chanussot,et al.  Fusion of Multispectral and Panchromatic Images Based on Morphological Operators , 2016, IEEE Transactions on Image Processing.

[62]  Jocelyn Chanussot,et al.  Indusion: Fusion of Multispectral and Panchromatic Images Using the Induction Scaling Technique , 2008, IEEE Geoscience and Remote Sensing Letters.

[63]  Manfred Ehlers,et al.  Performance of evaluation methods in image fusion , 2009, 2009 12th International Conference on Information Fusion.

[64]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[65]  Jocelyn Chanussot,et al.  Pansharpening Quality Assessment Using the Modulation Transfer Functions of Instruments , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[66]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[67]  Te-Ming Tu,et al.  Adjustable intensity-hue-saturation and Brovey transform fusion technique for IKONOS/ QuickBird imagery , 2005 .

[68]  Farhad Samadzadegan,et al.  Spectral and Spatial Quality assessment of IHS and Wavelet Based Pan-sharpening Techniques for High Resolution Satellite Imagery , 2018 .