Varying weighted spatial quality assessment for high resolution satellite image pan-sharpening

ABSTRACT This paper focuses on spatial quality assessment of pan-sharpened imagery that contains valuable information of input images. Its aim is to show that fusion functions respond differently to different types of landscapes. It compares a quality assessment of an object-level procedure with that of a conventional pixel-level-based procedure which assigns uniform quality scores to all image pixels of pan-sharpened images. To do so, after performing a series of pan-sharpening evaluations, a weighted procedure for spatial quality assessments of pan-sharpening products, allocating spatially varying weight factors to the image pixels proportional to their level of spatial information content is proposed. All experiments are performed using five high-resolution image datasets using fusion products produced by three common pan-sharpening algorithms. The datasets are acquired from WorldView-2, QuickBird, and IKONOS. Experimental results show that the spatial distortion of fused images for the class vegetation cover exceeds that of man-made structures, reaching more than 4% in some cases. Our procedure can preclude illogical fidelity estimations occurring when pan-sharpened images contain different land covers. Since particular image structures are of high importance in remote sensing applications, our procedure provides a purpose-oriented estimation of the spatial quality for pan-sharpened images in comparison with conventional procedures.

[1]  Gherardo Chirici,et al.  A meta-analysis and review of the literature on the k-Nearest Neighbors technique for forestry applications that use remotely sensed data , 2016 .

[2]  GilbertsonJason Kane,et al.  Effect of pan-sharpening multi-temporal Landsat 8 imagery for crop type differentiation using different classification techniques , 2017 .

[3]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[4]  A. Hegde,et al.  A Review of Quality Metrics for Fused Image , 2015 .

[5]  Isabel Luisa Castillejo-González,et al.  Mapping of Olive Trees Using Pansharpened QuickBird Images: An Evaluation of Pixel- and Object-Based Analyses , 2018, Agronomy.

[6]  Zaifeng Shi,et al.  Full-reference image quality assessment based on image segmentation with edge feature , 2018, Signal Process..

[7]  U. Benz,et al.  Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information , 2004 .

[8]  Gemine Vivone,et al.  Perceptual Quality Assessment of Pan-Sharpened Images , 2019, Remote. Sens..

[9]  Yuzhen Hong,et al.  A no-reference image blurriness metric in the spatial domain , 2016 .

[10]  Luciano Alparone,et al.  Spatial Consistency for Full-Scale Assessment of Pansharpening , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[11]  Josaphat Tetuko Sri Sumantyo,et al.  Assessment of pan-sharpening methods applied to image fusion of remotely sensed multi-band data , 2012, Int. J. Appl. Earth Obs. Geoinformation.

[12]  Jocelyn Chanussot,et al.  Pansharpening Quality Assessment Using the Modulation Transfer Functions of Instruments , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Farhad Samadzadegan,et al.  Object-based spectral quality assessment of high-resolution pan-sharpened satellite imageries: new combined fusion strategy to increase the spectral quality , 2020, Arabian Journal of Geosciences.

[14]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[15]  Qiuming Cheng,et al.  Two improvement schemes of PAN modulation fusion methods for spectral distortion minimization , 2009 .

[16]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Ali Mohammadzadeh,et al.  A Novel Radiometric Control Set Sample Selection Strategy for Relative Radiometric Normalization of Multitemporal Satellite Images , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Qiuming Cheng,et al.  An image fusion method for misaligned panchromatic and multispectral data , 2011 .

[19]  Lina J. Karam,et al.  A No-Reference Objective Image Sharpness Metric Based on the Notion of Just Noticeable Blur (JNB) , 2009, IEEE Transactions on Image Processing.

[20]  Mario Chica-Olmo,et al.  Downscaling cokriging for image sharpening , 2006 .

[21]  Jaco Kemp,et al.  Effect of pan-sharpening multi-temporal Landsat 8 imagery for crop type differentiation using different classification techniques , 2017, Comput. Electron. Agric..

[22]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[23]  Francesca Bovolo,et al.  Analysis of the Effects of Pansharpening in Change Detection on VHR Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[24]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[25]  H. B. Mitchell.  Image Fusion: Theories, Techniques and Applications , 2010 .

[26]  Gintautas Palubinskas,et al.  Joint Quality Measure for Evaluation of Pansharpening Accuracy , 2015, Remote. Sens..

[27]  Wolfgang Reinhardt,et al.  Image segmentation for the purpose of object-based classification , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[28]  Farhad Samadzadegan,et al.  Evaluating the Sensitivity of Image Fusion Quality Metrics to Image Degradation in Satellite Imagery , 2011 .

[29]  Farhad Samadzadegan,et al.  Spectral and Spatial Quality assessment of IHS and Wavelet Based Pan-sharpening Techniques for High Resolution Satellite Imagery , 2018 .

[30]  Linhai Jing,et al.  Improvement of a Pansharpening Method Taking Into Account Haze , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[31]  Onisimo Mutanga,et al.  Mapping forest aboveground biomass in the reforested Buffelsdraai landfill site using texture combinations computed from SPOT-6 pan-sharpened imagery , 2019, Int. J. Appl. Earth Obs. Geoinformation.

[32]  Zhou Wang,et al.  Introduction to the Issue on Visual Media Quality Assessment , 2009, IEEE J. Sel. Top. Signal Process..

[33]  K. Nikolakopoulos,et al.  Quality assessment of ten fusion techniques applied on Worldview-2 , 2015 .

[34]  Shuyuan Yang,et al.  Pan-sharpening via deep metric learning , 2018 .

[35]  Kai-Kuang Ma,et al.  Screen content image quality assessment using edge model , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[36]  Mario Chica-Olmo,et al.  DSCOKRI: A library of computer programs for downscaling cokriging in support of remote sensing applications , 2010, Comput. Geosci..

[37]  S. Baronti,et al.  Twenty-Five Years of Pansharpening: A Critical Review and New Developments , 2012 .

[38]  Martin Kappas,et al.  Comparison of Random Forest, k-Nearest Neighbor, and Support Vector Machine Classifiers for Land Cover Classification Using Sentinel-2 Imagery , 2017, Sensors.

[39]  Sophia Antipolis,et al.  Assessment of the quality of fused products , 2004 .

[40]  F. Samadzadegan,et al.  An object-level strategy for pan-sharpening quality assessment of high-resolution satellite imagery , 2014 .

[41]  Snehmani,et al.  A comparative analysis of pansharpening techniques on QuickBird and WorldView-3 images , 2017 .

[42]  Qingsheng Liu,et al.  Quality Assessment by Region and Land Cover of Sharpening Approaches Applied to GF-2 Imagery , 2020, Applied Sciences.

[43]  Sérgio Freire,et al.  Introducing mapping standards in the quality assessment of buildings extracted from very high resolution satellite imagery , 2014 .

[44]  Farhad Samadzadegan,et al.  A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery , 2021 .

[45]  Peter M. Atkinson,et al.  Modelling the semivariograms and cross-semivariograms required in downscaling cokriging by numerical convolution-deconvolution , 2007, Comput. Geosci..

[46]  Xiangchu Feng,et al.  Edge Strength Similarity for Image Quality Assessment , 2013, IEEE Signal Processing Letters.

[47]  Huchuan Lu,et al.  Saliency detection via Cellular Automata , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[49]  H. Gokhan Ilk,et al.  The effect of Laplacian filter in adaptive unsharp masking for infrared image enhancement , 2011 .

[50]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[51]  Hui Li,et al.  An Improved Pansharpening Method for Misaligned Panchromatic and Multispectral Data , 2018, Sensors.

[52]  Mahdi Hasanlou,et al.  Quality assessment of pan-sharpening methods in high-resolution satellite images using radiometric and geometric index , 2015, Arabian Journal of Geosciences.

[53]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[54]  A. Mohammadzadeh,et al.  Road extraction based on fuzzy logic and mathematical morphology from pan‐sharpened ikonos images , 2006 .

[55]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Wenzhong Shi,et al.  Information Loss-Guided Multi-Resolution Image Fusion , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[57]  J. Nichol,et al.  Satellite remote sensing for detailed landslide inventories using change detection and image fusion , 2005 .

[58]  Jocelyn Chanussot,et al.  Improving MODIS Spatial Resolution for Snow Mapping Using Wavelet Fusion and ARSIS Concept , 2008, IEEE Geoscience and Remote Sensing Letters.

[59]  Qiuming Cheng,et al.  An image fusion method taking into account phenological analogies and haze , 2011 .

[60]  Weisi Lin,et al.  Content-based image quality assessment using semantic information and luminance differences , 2014 .

[61]  Yun Zhang,et al.  An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images , 2005, Inf. Fusion.

[62]  Zhou Wang,et al.  Spatial Pooling Strategies for Perceptual Image Quality Assessment , 2006, 2006 International Conference on Image Processing.

[63]  A. Mohammadzadeh,et al.  New edge adaptive GIHS-BT-SFIM fusion method and class-based approach investigation , 2015 .

[64]  Peter Reinartz,et al.  Analysis and selection of pan-sharpening assessment measures , 2012 .

[65]  Gwendoline Blanchet,et al.  A Survey of Pansharpening Methods with A New Band-Decoupled Variational Model , 2016, ArXiv.

[66]  Weiwei Sun,et al.  Pansharpening for Cloud-Contaminated Very High-Resolution Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[67]  Jon Atli Benediktsson,et al.  Pansharpening With Matting Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[68]  Myeong-Ryong Nam,et al.  Fusion of multispectral and panchromatic Satellite images using the curvelet transform , 2005, IEEE Geoscience and Remote Sensing Letters.

[69]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[70]  Ying Li,et al.  A weighted full-reference image quality assessment based on visual saliency , 2017, J. Vis. Commun. Image Represent..

[71]  Yaoqin Xie,et al.  Applications of edge preservation ratio in image processing , 2014, 2014 12th International Conference on Signal Processing (ICSP).

[72]  Farhad Samadzadegan,et al.  Spatial Quality Assessment of Pan-Sharpened High Resolution Satellite Imagery Based on an Automatically Estimated Edge Based Metric , 2013, Remote. Sens..

[73]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Zhengyou Wang,et al.  A New Image Quality Assessment Algorithm based on SSIM and Multiple Regressions , 2015 .

[75]  Wenzhong Shi,et al.  Fusion of Sentinel-2 images , 2016 .

[76]  Farhad Samadzadegan,et al.  A REVIEW ON SPATIAL QUALITY ASSESSMENT METHODS FOR EVALUATION OF PAN-SHARPENED SATELLITE IMAGERY , 2019 .

[77]  A. Hegde,et al.  Evaluation of pan-sharpening methods for spatial and spectral quality , 2017 .

[78]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[79]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[80]  Firouz Abdullah Al-Wassai,et al.  A Novel Metric Approach Evaluation For The Spatial Enhancement Of Pan-Sharpened Images , 2012, ArXiv.

[81]  Francisco Eugenio,et al.  Object-based quality evaluation procedure for fused remote sensing imagery , 2017, Neurocomputing.

[83]  Manfred Ehlers,et al.  Multi-sensor image fusion for pansharpening in remote sensing , 2010 .

[84]  Manfred Ehlers,et al.  Performance of evaluation methods in image fusion , 2009, 2009 12th International Conference on Information Fusion.

[85]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[86]  Yun Zhang,et al.  METHODS FOR IMAGE FUSION QUALITY ASSESSMENT - A REVIEW, COMPARISON AND ANALYSIS , 2008 .

[87]  Hui Li,et al.  Assessment of Pansharpening Methods Applied to WorldView-2 Imagery Fusion , 2017, Sensors.

[88]  Jocelyn Chanussot,et al.  Context-Adaptive Pansharpening Based on Image Segmentation , 2017, IEEE Transactions on Geoscience and Remote Sensing.