Assessment of Hyperspectral Sharpening Methods for the Monitoring of Natural Areas Using Multiplatform Remote Sensing Imagery

The use of cutting-edge geospatial technologies to monitor ecosystems and the development of tailored tools for assessing such natural areas is a fundamental task. In this context, the growing availability of hyperspectral (HS) imagery from satellite and aerial platforms can provide valuable information for the sustainable management of ecosystems. However, in some cases, the spectral richness provided by HS sensors is at the expense of spatial quality. To alleviate this inconvenience, which can be critical to monitor some heterogeneous and mixed natural areas, a number of HS sharpening techniques have been developed to increase the spatial resolution while trying to preserve the spectral content. This image processing field has attracted the interest of the scientific community, and many research studies have been conducted to assess the performance of different HS sharpening algorithms. In the last decade, however, many comparative studies rely upon simulated data. In this work, the challenging application of sharpening methods in real situations using multiplatform or multisensor data is also addressed. Thus, experiments with real data have been conducted, in addition to a thorough assessment of HS sharpening techniques using simulated imagery in scenarios with different spatial resolution ratios and registration errors. In particular, airborne and satellite HS imageries have been pansharpened with drone, orthophotos, and satellite high spatial resolution data evaluating 11 fusion algorithms. After a comprehensive analysis, considering different visual and quantitative quality indicators, the algorithm characteristics have been summarized and the methods with higher performance and robustness have been identified.

[1]  Davide Cozzolino,et al.  Target-Adaptive CNN-Based Pansharpening , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Jocelyn Chanussot,et al.  Pansharpening of Hyperspectral images using spatial distortion optimization , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[3]  Shuyuan Yang,et al.  Multispectral and Hyperspectral Image Fusion Based on Group Spectral Embedding and Low-Rank Factorization , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network , 2018, Remote. Sens..

[5]  N. Taleb,et al.  Recent Trends in Satellite Image Pan- sharpening techniques , 2014 .

[6]  John van Genderen,et al.  Structuring contemporary remote sensing image fusion , 2015 .

[7]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[9]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[10]  Richard Bamler,et al.  Jointly sparse fusion of hyperspectral and multispectral imagery , 2013, 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS.

[11]  Liangpei Zhang,et al.  Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[12]  Jixian Zhang Multi-source remote sensing data fusion: status and trends , 2010 .

[13]  Qian Du,et al.  Performance evaluation for pan-sharpening techniques , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[14]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[15]  W. J. Carper,et al.  The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data , 1990 .

[16]  J. Zhou,et al.  A wavelet transform method to merge Landsat TM and SPOT panchromatic data , 1998 .

[17]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Shutao Li,et al.  Multispectral and hyperspectral image fusion with spatial-spectral sparse representation , 2019, Inf. Fusion.

[19]  Todd Wittman,et al.  A variational approach to hyperspectral image fusion , 2009, Defense + Commercial Sensing.

[20]  Lorenzo Bruzzone,et al.  Fusion of Hyperspectral and LIDAR Remote Sensing Data for Classification of Complex Forest Areas , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Russell C. Hardie,et al.  MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor , 2004, IEEE Transactions on Image Processing.

[22]  Mingyi He,et al.  Hyperspherical color transform based pansharpening method for WorldView-2 satellite images , 2013, 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA).

[23]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Johannes R. Sveinsson,et al.  Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[25]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[26]  Peijun Du,et al.  Hyperspectral Remote Sensing Image Classification Based on Rotation Forest , 2014, IEEE Geoscience and Remote Sensing Letters.

[27]  E. Miguel,et al.  THE PROCESSING OF CASI-1500I DATA AT INTA PAF , 2014 .

[28]  Luciano Alparone,et al.  Hyperspectral image fusion , 2010 .

[29]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .

[30]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[31]  J. A. Schell,et al.  Monitoring vegetation systems in the great plains with ERTS , 1973 .

[32]  A. Mookambiga,et al.  Comprehensive review on fusion techniques for spatial information enhancement in hyperspectral imagery , 2016, Multidimens. Syst. Signal Process..

[33]  Jocelyn Chanussot,et al.  MultiResolution Analysis and Component Substitution techniques for hyperspectral Pansharpening , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[34]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[35]  Junfeng Yang,et al.  PanNet: A Deep Network Architecture for Pan-Sharpening , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[36]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Bin Wang,et al.  Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods , 2014, IEEE Geoscience and Remote Sensing Letters.

[38]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Russell C. Hardie,et al.  Application of the stochastic mixing model to hyperspectral resolution enhancement , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[40]  T. Chan,et al.  Fast dual minimization of the vectorial total variation norm and applications to color image processing , 2008 .

[41]  Jungho Im,et al.  ISPRS Journal of Photogrammetry and Remote Sensing , 2022 .

[42]  Javier Marcello,et al.  Evaluation of Spatial and Spectral Effectiveness of Pixel-Level Fusion Techniques , 2013, IEEE Geoscience and Remote Sensing Letters.

[43]  Qian Du,et al.  Hyperspectral and LiDAR Data Fusion: Outcome of the 2013 GRSS Data Fusion Contest , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[44]  Jocelyn Chanussot,et al.  Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data , 2014, IEEE Transactions on Image Processing.

[45]  Xiaohuan Xi,et al.  Fusion of Airborne Discrete-Return LiDAR and Hyperspectral Data for Land Cover Classification , 2015, Remote. Sens..

[46]  Davide Cozzolino,et al.  Pansharpening by Convolutional Neural Networks , 2016, Remote. Sens..

[47]  Bruno Aiazzi,et al.  Hyper-Sharpening: A First Approach on SIM-GA Data , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[48]  X Li Band grouping pansharpening for WorldView-2 satellite images , 2013 .

[49]  Christine Pohl CHALLENGES OF REMOTE SENSING IMAGE FUSION TO OPTIMIZE EARTH OBSERVATION DATA EXPLOITATION , 2014 .

[50]  Pedram Ghamisi,et al.  Hyperspectral and LiDAR Fusion Using Extinction Profiles and Total Variation Component Analysis , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Hongyi Liu,et al.  A New Pan-Sharpening Method With Deep Neural Networks , 2015, IEEE Geoscience and Remote Sensing Letters.

[52]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[53]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[54]  Xiuping Jia,et al.  A Novel Adversarial Based Hyperspectral and Multispectral Image Fusion , 2019, Remote. Sens..

[55]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[56]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[57]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[58]  Fred A. Kruse,et al.  The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data , 1993 .

[59]  Jean-Yves Tourneret,et al.  Bayesian fusion of multispectral and hyperspectral images using a block coordinate descent method , 2015, 2015 7th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).