A large-scale remote sensing database for subjective and objective quality assessment of pansharpened images

Abstract Pansharpening is a process to fuse a low spatial resolution multispectral image and a high spatial resolution panchromatic image to produce a high-resolution multispectral image. Quality assessment of pansharpened images is challenging due to without actual reference images. There are two main types of assessment methods: reduced resolution (RR) assessment based on Wald’s protocol, and full resolution (FR) assessment without reference. Currently, it is lack of large-scale benchmark databases for subjective and objective performance evaluation of different image pansharpening methods. In this paper, we construct a large-scale database named Pansharpened Remote Sensing Image Quality Database (PRSIQD) from both qualitative and quantitative perspectives, which contains 13,620 pansharpened images acquired from IKONOS, QuickBird, Gaofen-1, WorldView-2, WorldView-3 and WorldView-4 satellite sensors. In addition, we have comprehensively analyzed the advantages and disadvantages of the existing pansharpening quality assessment methods on different satellite sensors, thematic datasets and bands.

[1]  Edurne Ibarrola-Ulzurrun,et al.  Influence of Pansharpening in Obtaining Accurate Vegetation Maps , 2017 .

[2]  Gemine Vivone,et al.  Perceptual Quality Assessment of Pan-Sharpened Images , 2019, Remote. Sens..

[3]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[5]  Liangpei Zhang,et al.  An Integrated Framework for the Spatio–Temporal–Spectral Fusion of Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Jaco Kemp,et al.  Effect of pan-sharpening multi-temporal Landsat 8 imagery for crop type differentiation using different classification techniques , 2017, Comput. Electron. Agric..

[7]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[8]  Jocelyn Chanussot,et al.  A Combiner-Based Full Resolution Quality Assessment Index for Pansharpening , 2019, IEEE Geoscience and Remote Sensing Letters.

[9]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[10]  Zhuo Chen,et al.  Unified No-Reference Quality Assessment of Singly and Multiply Distorted Stereoscopic Images , 2019, IEEE Transactions on Image Processing.

[11]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Johannes R. Sveinsson,et al.  Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[14]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[15]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[16]  Francesca Bovolo,et al.  Analysis of the Effects of Pansharpening in Change Detection on VHR Images , 2010, IEEE Geoscience and Remote Sensing Letters.

[17]  Weiwei Sun,et al.  Pansharpening for Cloud-Contaminated Very High-Resolution Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Jocelyn Chanussot,et al.  A Bayesian Procedure for Full-Resolution Quality Assessment of Pansharpened Products , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[21]  Jocelyn Chanussot,et al.  Improving MODIS Spatial Resolution for Snow Mapping Using Wavelet Fusion and ARSIS Concept , 2008, IEEE Geoscience and Remote Sensing Letters.

[22]  Andrea Garzelli,et al.  Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis , 2002, IEEE Trans. Geosci. Remote. Sens..

[23]  Laura Igual,et al.  A Variational Model for P+XS Image Fusion , 2006, International Journal of Computer Vision.

[24]  P. Chavez,et al.  Extracting spectral contrast in landsat thematic mapper image data using selective principal component analysis , 1989 .

[25]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[26]  Michael Dixon,et al.  Google Earth Engine: Planetary-scale geospatial analysis for everyone , 2017 .

[27]  Farhad Samadzadegan,et al.  Spatial Quality Assessment of Pan-Sharpened High Resolution Satellite Imagery Based on an Automatically Estimated Edge Based Metric , 2013, Remote. Sens..

[28]  Brian Johnson,et al.  Effects of Pansharpening on Vegetation Indices , 2014, ISPRS Int. J. Geo Inf..

[29]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[30]  Andrea Garzelli,et al.  Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[31]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[32]  Francesca Bovolo,et al.  Updating Land-Cover Maps by Classification of Image Time Series: A Novel Change-Detection-Driven Transfer Learning Approach , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Jinying Zhong,et al.  Remote Sensing Image Fusion with Convolutional Neural Network , 2016 .

[34]  Liangpei Zhang,et al.  Quality Assessment of Panchromatic and Multispectral Image Fusion for the ZY-3 Satellite: From an Information Extraction Perspective , 2014, IEEE Geoscience and Remote Sensing Letters.

[35]  Luciano Alparone,et al.  A global quality measurement of pan-sharpened multispectral imagery , 2004, IEEE Geoscience and Remote Sensing Letters.

[36]  L. Wald,et al.  Fusion of high spatial and spectral resolution images : The ARSIS concept and its implementation , 2000 .

[37]  Andrea Garzelli,et al.  Hypercomplex Quality Assessment of Multi/Hyperspectral Images , 2009, IEEE Geoscience and Remote Sensing Letters.

[38]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Luciano Alparone,et al.  A Theoretical Analysis of the Effects of Aliasing and Misregistration on Pansharpened Imagery , 2011, IEEE Journal of Selected Topics in Signal Processing.

[40]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[41]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[42]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[43]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[46]  Robert A. Schowengerdt,et al.  Reconstruction of multispatial, multispectral image data using spatial frequency content , 1980 .

[47]  Gintautas Palubinskas,et al.  Joint Quality Measure for Evaluation of Pansharpening Accuracy , 2015, Remote. Sens..

[48]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[49]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[50]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[51]  Chiman Kwan,et al.  Blind Quality Assessment of Fused WorldView-3 Images by Using the Combinations of Pansharpening and Hypersharpening Paradigms , 2017, IEEE Geoscience and Remote Sensing Letters.

[52]  Francisco Eugenio,et al.  Object-based quality evaluation procedure for fused remote sensing imagery , 2017, Neurocomputing.

[53]  Ming Zhong,et al.  Object-Based Classification of Urban Areas Using VHR Imagery and Height Points Ancillary Data , 2012, Remote. Sens..

[54]  Mahdi Hasanlou,et al.  Quality assessment of pan-sharpening methods in high-resolution satellite images using radiometric and geometric index , 2015, Arabian Journal of Geosciences.

[55]  Feng Shao,et al.  No-Reference Quality Assessment for Pansharpened Images via Opinion-Unaware Learning , 2019, IEEE Access.

[56]  Nikolay N. Ponomarenko,et al.  TID2008 – A database for evaluation of full-reference visual quality assessment metrics , 2004 .

[57]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[58]  Stefano Baronti,et al.  On the Use of the Expanded Image in Quality Assessment of Pansharpened Images , 2018, IEEE Geoscience and Remote Sensing Letters.