Fusing China GF-5 Hyperspectral Data with GF-1, GF-2 and Sentinel-2A Multispectral Data: Which Methods Should Be Used?

The China GaoFen-5 (GF-5) satellite sensor, which was launched in 2018, collects hyperspectral data with 330 spectral bands, a 30 m spatial resolution, and 60 km swath width. Its competitive advantages compared to other on-orbit or planned sensors are its number of bands, spectral resolution, and swath width. Unfortunately, its applications may be undermined by its relatively low spatial resolution. Therefore, the data fusion of GF-5 with high spatial resolution multispectral data is required to further enhance its spatial resolution while preserving its spectral fidelity. This paper conducted a comprehensive evaluation study of fusing GF-5 hyperspectral data with three typical multispectral data sources (i.e., GF-1, GF-2 and Sentinel-2A (S2A)), based on quantitative metrics, classification accuracy, and computational efficiency. Datasets on three study areas of China were utilized to design numerous experiments, and the performances of nine state-of-the-art fusion methods were compared. Experimental results show that LANARAS (this method was proposed by lanaras et al.), Adaptive Gram–Schmidt (GSA), and modulation transfer function (MTF)-generalized Laplacian pyramid (GLP) methods are more suitable for fusing GF-5 with GF-1 data, MTF-GLP and GSA methods are recommended for fusing GF-5 with GF-2 data, and GSA and smoothing filtered-based intensity modulation (SFIM) can be used to fuse GF-5 with S2A data.

[1]  K. P. Soman,et al.  Implementation and Comparative Study of Image Fusion Algorithms , 2010 .

[2]  Antonio J. Plaza,et al.  Parallel real-time virtual dimensionality estimation for hyperspectral images , 2017, Journal of Real-Time Image Processing.

[3]  Michael Unser,et al.  A note on cubic convolution interpolation , 2003, IEEE Trans. Image Process..

[4]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network , 2018, Remote. Sens..

[5]  V. Raghu Venkataraman,et al.  Reflectance based vicarious calibration of HySIS sensors and spectral stability study over pseudo-invariant sites , 2019, 2019 IEEE Recent Advances in Geoscience and Remote Sensing : Technologies, Standards and Applications (TENGARSS).

[6]  Liangpei Zhang,et al.  An Integrated Framework for the Spatio–Temporal–Spectral Fusion of Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[7]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[8]  Xin Wu,et al.  GPU-Based Parallel Design of the Hyperspectral Signal Subspace Identification by Minimum Error (HySime) , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Stephen V. Stehman,et al.  Selecting and interpreting measures of thematic classification accuracy , 1997 .

[10]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Johannes R. Sveinsson,et al.  Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[12]  Xiande Liu,et al.  Bilinear interpolation method for quantum images based on quantum Fourier transform , 2018, International Journal of Quantum Information.

[13]  Wen-jiang Huang,et al.  [Using hyperspectral remote sensing to estimate canopy chlorophyll density of wheat under yellow rust stress]. , 2010, Guang pu xue yu guang pu fen xi = Guang pu.

[14]  Stephen P. Boyd,et al.  An ADMM Algorithm for a Class of Total Variation Regularized Estimation Problems , 2012, 1203.1828.

[15]  Stefania Matteoli,et al.  Environmental products overview of the Italian hyperspectral prisma mission: The SAP4PRISMA project , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[16]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral Image Super-Resolution Based on Spatial and Spectral Correlation Fusion , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[17]  José M. P. Nascimento,et al.  Parallel Hyperspectral Unmixing Method via Split Augmented Lagrangian on GPU , 2016, IEEE Geoscience and Remote Sensing Letters.

[18]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[20]  Paul E. Lewis,et al.  MODTRAN4-based atmospheric correction algorithm: FLAASH (fast line-of-sight atmospheric analysis of spectral hypercubes) , 2002, SPIE Defense + Commercial Sensing.

[21]  P. Thenkabail,et al.  Hyperspectral Vegetation Indices and Their Relationships with Agricultural Crop Characteristics , 2000 .

[22]  S. Ganguly,et al.  Sparse unmixing via variable splitting and augmented Lagrangian for vegetation and urban area classification using Landsat data , 2015 .

[23]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[24]  Russell C. Hardie,et al.  Application of the stochastic mixing model to hyperspectral resolution enhancement , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Akira Iwasaki,et al.  Current status of Hyperspectral Imager Suite (HISUI) onboard International Space Station (ISS) , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[26]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[27]  U. Benz,et al.  The EnMAP hyperspectral imager—An advanced optical payload for future applications in Earth observation programmes , 2006 .

[28]  Huang Wei,et al.  GF‐2 Satellite 1m/4m Camera Design and In‐Orbit Commissioning , 2018, Chinese Journal of Electronics.

[29]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[30]  Chiman Kwan,et al.  Hyperspectral image super-resolution: a hybrid color mapping approach , 2016 .

[31]  Arif Mahmood,et al.  Hyperspectral Face Recognition With Spatiospectral Information Fusion and PLS Regression , 2015, IEEE Transactions on Image Processing.

[32]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[33]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[34]  Jean-Yves Tourneret,et al.  Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation , 2015, IEEE Transactions on Image Processing.

[35]  A. Amanatiadis,et al.  A survey on evaluation methods for image interpolation , 2009 .

[36]  Uta Heiden,et al.  THE NEW HYPERSPECTRAL SENSOR DESIS ON THE MULTI-PAYLOAD PLATFORM MUSES INSTALLED ON THE ISS , 2016 .

[37]  Weiwei Sun,et al.  Pansharpening for Cloud-Contaminated Very High-Resolution Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Sheila S. Hemami,et al.  Regularity-preserving image interpolation , 1999, IEEE Trans. Image Process..

[39]  Wenzhong Shi,et al.  Fusion of Sentinel-2 images , 2016 .

[40]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Yao Liu,et al.  Land Surface Temperature Estimate From Chinese Gaofen-5 Satellite Data Using Split-Window Algorithm , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[43]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Roger L. King,et al.  An Efficient Pan-Sharpening Method via a Combined Adaptive PCA Approach and Contourlets , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Bruno Aiazzi,et al.  Hyper-Sharpening: A First Approach on SIM-GA Data , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[46]  S. M. Mahbubur Rahman,et al.  Image fusion technique using multivariate statistical model for wavelet coefficients , 2013, Signal Image Video Process..

[47]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[48]  Yifan Zhang Wavelet-based Bayesian fusion of multispectral and hyperspectral images using Gaussian scale mixture model , 2012 .

[49]  Michael Möller,et al.  An Adaptive IHS Pan-Sharpening Method , 2010, IEEE Geoscience and Remote Sensing Letters.

[50]  Yimin Wei,et al.  Condition Numbers of the Generalized Sylvester Equation , 2007, IEEE Transactions on Automatic Control.

[51]  Brian M. Steele,et al.  Maximum posterior probability estimators of map accuracy , 2005 .

[52]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[54]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[55]  Liangpei Zhang,et al.  An Adaptive Mean-Shift Analysis Approach for Object Extraction and Classification From Urban Hyperspectral Imagery , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[56]  LiShutao,et al.  Pixel-level image fusion , 2017 .

[57]  Hanqiang Cao,et al.  Nearest Neighbor Value Interpolation , 2012, ArXiv.

[58]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[59]  Jay Gao,et al.  Hyperspectral sensing of heavy metals in soil and vegetation: Feasibility and challenges , 2018 .

[60]  Zhaoguang,et al.  GF-1 Satellite——The First Satellite of CHEOS , 2013 .