On the validation of pansharpening methods

Validation of the quality of pansharpening methods is a difficult task because the reference is not directly available. In the meantime, two main approaches have been established: validation in reduced resolution and original resolution. In the former approach it is still not clear how the data are to be processed to a lower resolution. Other open issues are related to the question which resolution and measures should be used. In the latter approach the main problem is how the appropriate measure should be selected. In the most comparison studies the results of both approaches do not correspond, that means in each case other methods are selected as the best ones. Thus, the developers of the new pansharpening methods still stand in the front of dilemma: how to perform a correct or appropriate comparison/evaluation/validation. It should be noted, that the third approach is possible, that is to perform the comparison of methods in a particular application with the usage of their ground truth. But this is not always possible, because usually developers are not working with applications. Moreover, it can be an additional computational load for a researcher in a particular application. In this paper some of the questions/problems raised above are approached/discussed. The following component substitution (CS) and high pass filtering (HPF) pansharpening methods with additive and multiplicative models and their enhancements such as haze correction, histogram matching, usage of spectral response functions (SRF), modulation transfer function (MTF) based lowpass filtering are investigated on remote sensing data of WorldView-2 and WorldView-4 sensors.

[1]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[2]  Luciano Alparone,et al.  Fast Reproducible Pansharpening Based on Instrument and Acquisition Modeling: AWLP Revisited , 2019, Remote. Sens..

[3]  Peter Reinartz,et al.  Multi-resolution, multi-sensor image fusion: general fusion framework , 2011, 2011 Joint Urban Remote Sensing Event.

[4]  Gintautas Palubinskas,et al.  Fast, simple, and good pan-sharpening method , 2013 .

[5]  Xavier Otazu,et al.  A low computational-cost method to fuse IKONOS images using the spectral response function of its sensors , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[6]  S. Baronti,et al.  Multispectral and panchromatic data fusion assessment without reference , 2008 .

[7]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[8]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[9]  G. Palubinskas PAN-SHARPENING APPROACHES BASED ON UNMIXING OF MULTISPECTRAL REMOTE SENSING IMAGERY , 2016 .

[10]  Luciano Alparone,et al.  Intersensor Statistical Matching for Pansharpening: Theoretical Issues and Practical Solutions , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[12]  Gintautas Palubinskas,et al.  Image similarity/distance measures: what is really behind MSE and SSIM? , 2017 .

[13]  Dieter Oertel,et al.  Unmixing-based multisensor multiresolution image fusion , 1999, IEEE Trans. Geosci. Remote. Sens..

[14]  Altan Mesut,et al.  A comparative analysis of image fusion methods , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[15]  Yongsun Kim,et al.  Generalized IHS-Based Satellite Imagery Fusion Using Spectral Response Functions , 2011 .

[16]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Hongyan Zhang,et al.  A Large-Scale Benchmark Data Set for Evaluating Pansharpening Performance: Overview and Implementation , 2020, IEEE Geoscience and Remote Sensing Magazine.

[18]  Luciano Alparone,et al.  Haze Correction for Contrast-Based Multispectral Pansharpening , 2017, IEEE Geoscience and Remote Sensing Letters.

[19]  Christine Pohl,et al.  Remote Sensing Image Fusion: A Practical Guide , 2016 .

[20]  Jocelyn Chanussot,et al.  A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening With Classical and Emerging Pansharpening Methods , 2021, IEEE Geoscience and Remote Sensing Magazine.

[21]  Luciano Alparone,et al.  Multispectral Pansharpening with Radiative Transfer-Based Detail-Injection Modeling for Preserving Changes in Vegetation Cover , 2018, Remote. Sens..

[22]  Peter Reinartz,et al.  MULTIRESOLUTION IMAGE FUSION: PHASE CONGRUENCYFOR SPATIAL CONSISTENCY ASSESSMENT , 2010 .

[23]  P. Stark Bounded-Variable Least-Squares: an Algorithm and Applications , 2008 .

[24]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[25]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Gintautas Palubinskas,et al.  Model-based view at multi-resolution image fusion methods and quality assessment measures , 2016 .

[27]  Gintautas Palubinskas,et al.  Model-based image adjustment for a successful pansharpening , 2021, ArXiv.

[28]  Johannes R. Sveinsson,et al.  Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Peter Reinartz,et al.  Analysis and selection of pan-sharpening assessment measures , 2012 .

[30]  Aggelos K. Katsaggelos,et al.  A survey of classical methods and new trends in pansharpening of multispectral images , 2011, EURASIP J. Adv. Signal Process..

[31]  Johannes R. Sveinsson,et al.  Model-Based Satellite Image Fusion , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Peter Reinartz,et al.  Multi-sensor data fusion for urban area classification , 2011, 2011 Joint Urban Remote Sensing Event.

[34]  Xavier Otazu,et al.  Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Manfred Ehlers,et al.  Multi-sensor image fusion for pansharpening in remote sensing , 2010 .

[36]  Luciano Alparone,et al.  Quality assessment of pansharpening methods and products , 2011 .

[37]  Krištof Oštir,et al.  High-resolution image fusion : Methods to preserve spectral and spatial resolution , 2006 .

[38]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[39]  John van Genderen,et al.  Structuring contemporary remote sensing image fusion , 2015 .

[40]  Gintautas Palubinskas Mystery behind similarity measures mse and SSIM , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[41]  Gintautas Palubinskas,et al.  Joint Quality Measure for Evaluation of Pansharpening Accuracy , 2015, Remote. Sens..

[42]  S. Baronti,et al.  Remote Sensing Image Fusion , 2015 .

[43]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.