Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability

Image fusion combines data from different heterogeneous sources to obtain more precise information about an underlying scene. Hyperspectral-multispectral (HS-MS) image fusion is currently attracting great interest in remote sensing since it allows the generation of high spatial resolution HS images and circumventing the main limitation of this imaging modality. Existing HS-MS fusion algorithms, however, neglect the spectral variability often existing between images acquired at different time instants. This time difference causes variations in spectral signatures of the underlying constituent materials due to the different acquisition and seasonal conditions. This paper introduces a novel HS-MS image fusion strategy that combines an unmixing-based formulation with an explicit parametric model for typical spectral variability between the two images. Simulations with synthetic and real data show that the proposed strategy leads to a significant performance improvement under spectral variability and state-of-the-art performance otherwise.

[1]  D. Roy,et al.  Multi-temporal MODIS-Landsat data fusion for relative radiometric normalization, gap filling, and prediction of Landsat data , 2008 .

[2]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[3]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[4]  Ricardo Augusto Borsoi,et al.  A Low-Rank Tensor Regularization Strategy for Hyperspectral Unmixing , 2018, 2018 IEEE Statistical Signal Processing Workshop (SSP).

[5]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[6]  John F. Arnold,et al.  Reliably estimating the noise in AVIRIS hyperspectral images , 1996 .

[7]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[8]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[11]  W. Verstraeten,et al.  Nonlinear Hyperspectral Mixture Analysis for tree cover estimates in orchards , 2009 .

[12]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[13]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Ajmal S. Mian,et al.  Bayesian sparse representation for hyperspectral image super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jean-Yves Tourneret,et al.  Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation , 2015, IEEE Transactions on Image Processing.

[16]  Tim R. McVicar,et al.  Assessing the accuracy of blending Landsat–MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection , 2013 .

[17]  Nikos D. Sidiropoulos,et al.  Hyperspectral Super-Resolution Via Coupled Tensor Factorization: Identifiability and Algorithms , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  Yi Ma,et al.  A non-negative sparse promoting algorithm for high resolution hyperspectral imaging , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Jocelyn Chanussot,et al.  Blind Hyperspectral Unmixing Using an Extended Linear Mixing Model to Address Spectral Variability , 2016, IEEE Transactions on Image Processing.

[20]  Jocelyn Chanussot,et al.  Spectral Unmixing: A Derivation of the Extended Linear Mixing Model From the Hapke Model , 2019, IEEE Geoscience and Remote Sensing Letters.

[21]  Russell C. Hardie,et al.  MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor , 2004, IEEE Transactions on Image Processing.

[22]  Joanne C. White,et al.  Generation of dense time series synthetic Landsat data through data blending with MODIS using a spatial and temporal adaptive reflectance fusion model. , 2009 .

[23]  Gary A. Shaw,et al.  Spectral Imaging for Remote Sensing , 2003 .

[24]  Mathew R. Schwaller,et al.  On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Cédric Richard,et al.  A Fast Multiscale Spatial Regularization for Sparse Hyperspectral Unmixing , 2017, IEEE Geoscience and Remote Sensing Letters.

[26]  Alfonso Fernández-Manso,et al.  Spectral unmixing , 2012 .

[27]  Richard J. Murphy,et al.  A Novel Endmember Bundle Extraction and Clustering Approach for Capturing Spectral Variability Within Endmember Classes , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Shutao Li,et al.  Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization , 2018, IEEE Transactions on Image Processing.

[29]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[30]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[31]  Jean-Yves Tourneret,et al.  Hyperspectral Unmixing With Spectral Variability Using a Perturbed Linear Mixing Model , 2015, IEEE Transactions on Signal Processing.

[32]  Ricardo Augusto Borsoi,et al.  Generalized Linear Mixing Model Accounting for Endmember Variability , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Joanne C. White,et al.  A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS , 2009 .

[34]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[35]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Timo Stuffler,et al.  EnMAP A Hyperspectral Sensor for Environmental Mapping and Analysis , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[37]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[38]  David Krutz,et al.  DESIS (DLR Earth Sensing Imaging Spectrometer for the ISS-MUSES platform) , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[39]  K. C. Ho,et al.  Endmember Variability in Hyperspectral Analysis: Addressing Spectral Variability During Spectral Unmixing , 2014, IEEE Signal Processing Magazine.

[40]  Naoto Yokoya,et al.  Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data , 2014, IEEE Transactions on Image Processing.

[41]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[42]  Dimitris G. Manolakis,et al.  Detection algorithms for hyperspectral imaging applications , 2002, IEEE Signal Process. Mag..

[43]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.