Fusion of Hyperspectral and Multispectral Images Using Spectral Unmixing and Sparse Coding

Unlike multispectral (MSI) and panchromatic (PAN) images, generally the spatial resolution of hyperspectral images (HSI) is limited, due to sensor limitations. In many applications, HSI with a high spectral as well as spatial resolution are required. In this paper, a new method for spatial resolution enhancement of a HSI using spectral unmixing and sparse coding (SUSC) is introduced. The proposed method fuses high spectral resolution features from the HSI with high spatial resolution features from an MSI of the same scene. Endmembers are extracted from the HSI by spectral unmixing, and the exact location of the endmembers is obtained from the MSI. This fusion process by using spectral unmixing is formulated as an ill-posed inverse problem which requires a regularization term in order to convert it into a well-posed inverse problem. As a regularizer, we employ sparse coding (SC), for which a dictionary is constructed using high spatial resolution MSI or PAN images from unrelated scenes. The proposed algorithm is applied to real Hyperion and ROSIS datasets. Compared with other state-of-the-art algorithms based on pansharpening, spectral unmixing, and SC methods, the proposed method is shown to significantly increase the spatial resolution while perserving the spectral content of the HSI.

[1]  Ye Zhang,et al.  Integration of Spatial–Spectral Information for Resolution Enhancement in Hyperspectral Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[2]  José M. Bioucas-Dias,et al.  Minimum Volume Simplex Analysis: A Fast Algorithm to Unmix Hyperspectral Data , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[3]  Richard Bamler,et al.  Jointly sparse fusion of hyperspectral and multispectral imagery , 2013, 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS.

[4]  Bruno A. Olshausen,et al.  Learning Sparse Codes for Hyperspectral Imagery , 2011, IEEE Journal of Selected Topics in Signal Processing.

[5]  Lucien Wald,et al.  Quality of high resolution synthesised images: Is there a simple criterion ? , 2000 .

[6]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Chong-Yung Chi,et al.  A convex analysis-based minimum-volume enclosing simplex algorithm for hyperspectral unmixing , 2009, IEEE Trans. Signal Process..

[8]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[9]  Jocelyn Chanussot,et al.  Super-resolution of hyperspectral images using local spectral unmixing , 2014, 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[10]  Yücel Altunbasak,et al.  Super-resolution reconstruction of hyperspectral images , 2005 .

[11]  P. Reinartz,et al.  HYPERSPECTRAL IMAGE RESOLUTION ENHANCEMENT BASED ON SPECTRAL UNMIXING AND INFORMATION FUSION , 2012 .

[12]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[13]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[14]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[15]  Wei Xia,et al.  An approach based on constrained nonnegative matrix factorization to unmix hyperspectral data , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral Imagery Super-Resolution by Spatial–Spectral Joint Nonlocal Similarity , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Xiaoxiang Zhu,et al.  Towards a combined sparse representation and unmixing based hybrid hyperspectral resolution enhancement method , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[18]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[19]  Rob Heylen,et al.  Hyperspectral image noise reduction and its effect on spectral unmixing , 2014, 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[20]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[21]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[22]  José M. Bioucas-Dias,et al.  Hyperspectral Subspace Identification , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[23]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[25]  John F. Mustard,et al.  Spectral unmixing , 2002, IEEE Signal Process. Mag..

[26]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[27]  José M. Bioucas-Dias,et al.  Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[28]  Shaohui Mei,et al.  Hyperspectral Image Resolution Enhancement Using High-Resolution Multispectral Image Based on Spectral Unmixing , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[30]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[31]  S. Sides,et al.  Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic , 1991 .

[32]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[33]  Mario Winter,et al.  N-FINDR: an algorithm for fast autonomous spectral end-member determination in hyperspectral data , 1999, Optics & Photonics.

[34]  Michael Elad,et al.  Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit , 2008 .

[35]  José M. Bioucas-Dias,et al.  A variable splitting augmented Lagrangian approach to linear spectral unmixing , 2009, 2009 First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[36]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[37]  Ajmal S. Mian,et al.  Sparse Spatio-spectral Representation for Hyperspectral Image Super-resolution , 2014, ECCV.

[38]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Mohamed-Jalal Fadili,et al.  The Undecimated Wavelet Decomposition and its Reconstruction , 2007, IEEE Transactions on Image Processing.

[40]  Quan Pan,et al.  Hyperspectral imagery super-resolution by sparse representation and spectral regularization , 2011, EURASIP J. Adv. Signal Process..