Hyperspectral Super-Resolution via Interpretable Block-Term Tensor Modeling

Hyperspectral super-resolution (HSR) aims at fusing a pair of hyperspectral and multispectral images to recover a super-resolution image (SRI). This work revisits coupled tensor decomposition (CTD)-based HSR. The vast majority of the HSR approaches take a low-rank matrix recovery perspective. The challenge is that theoretical guarantees for recovering the SRI using low-rank matrix models are either elusive or derived under stringent conditions. A couple of recent CTD-based methods ensure recoverability for the SRI under relatively mild conditions, leveraging algebraic properties of the <italic>canonical polyadic decomposition</italic> (CPD) and the <italic>Tucker</italic> decomposition models, respectively. However, the latent factors of both the CPD and Tucker models have no physical interpretations in the context of spectral image analysis, which makes incorporating prior information challenging—but using priors is often essential for enhancing performance in noisy environments. This work employs an idea that models spectral images as tensors following the block-term decomposition model with multilinear rank-<inline-formula><tex-math notation="LaTeX">$(L_r,L_r,1)$</tex-math></inline-formula> terms (i.e., the <inline-formula><tex-math notation="LaTeX">${\mathsf {LL1}}$</tex-math></inline-formula> model) and formulates the HSR problem as a coupled <inline-formula><tex-math notation="LaTeX">${\mathsf {LL1}}$</tex-math></inline-formula> tensor decomposition problem. Similar to the existing CTD approaches, recoverability of the SRI is shown under mild conditions. More importantly, the latent factors of the <inline-formula><tex-math notation="LaTeX">${\mathsf {LL1}}$</tex-math></inline-formula> model can be interpreted as the key constituents of spectral images, i.e., the endmembers’ spectral signatures and abundance maps. This connection allows us to incorporate prior information for performance enhancement. A flexible algorithmic framework that can work with a series of structural information is proposed to take advantages of the model interpretability. The effectiveness is showcased using simulated and real data.

[1]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Simon J. Godsill,et al.  Multiband Image Fusion Based on Spectral Unmixing , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Yuan Yan Tang,et al.  Hyperspectral Unmixing via Total Variation Regularized Nonnegative Tensor Factorization , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Kejun Huang,et al.  Hyperspectral Super-Resolution: A Coupled Nonnegative Block-Term Tensor Decomposition Approach , 2019, 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[5]  Yuan Yan Tang,et al.  Matrix-Vector Nonnegative Tensor Factorization for Blind Unmixing of Hyperspectral Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Jean-Yves Tourneret,et al.  Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation , 2015, IEEE Transactions on Image Processing.

[7]  Jessica A. Faust,et al.  Imaging Spectroscopy and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) , 1998 .

[8]  Qiang Li,et al.  Hyperspectral Super-Resolution: Exact Recovery In Polynomial Time , 2018, 2018 IEEE Statistical Signal Processing Workshop (SSP).

[9]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[10]  M. F. Baumgardner,et al.  220 Band AVIRIS Hyperspectral Image Data Set: June 12, 1992 Indian Pine Test Site 3 , 2015 .

[11]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part I: Lemmas for Partitioned Matrices , 2008, SIAM J. Matrix Anal. Appl..

[13]  Qiang Li,et al.  Hyperspectral Super-Resolution via Global–Local Low-Rank Matrix Estimation , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[15]  Maryam Fazel,et al.  Iterative reweighted algorithms for matrix rank minimization , 2012, J. Mach. Learn. Res..

[16]  Gaofeng Meng,et al.  Spectral Unmixing via Data-Guided Sparsity , 2014, IEEE Transactions on Image Processing.

[17]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Lieven De Lathauwer,et al.  Coupled Canonical Polyadic Decompositions and (Coupled) Decompositions in Multilinear Rank-(Lr, n, Lr, n, 1) Terms - Part I: Uniqueness , 2015, SIAM J. Matrix Anal. Appl..

[19]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[20]  Robert F. Cromp,et al.  Support Vector Machine Classifiers as Applied to AVIRIS Data , 1999 .

[21]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[23]  L. Wald,et al.  Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images , 1997 .

[24]  Mehmet Akçakaya,et al.  Tensor Completion From Regular Sub-Nyquist Samples , 2019, IEEE Transactions on Signal Processing.

[25]  Shutao Li,et al.  Hyperspectral Image Super-Resolution via Non-local Sparse Tensor Factorization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Nikos D. Sidiropoulos,et al.  Joint Tensor Factorization and Outlying Slab Suppression With Applications , 2015, IEEE Transactions on Signal Processing.

[27]  Shutao Li,et al.  Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization , 2018, IEEE Transactions on Image Processing.

[28]  Yin Zhang,et al.  A Compressive Sensing and Unmixing Scheme for Hyperspectral Data Processing , 2012, IEEE Transactions on Image Processing.

[29]  Yi Ma,et al.  A non-negative sparse promoting algorithm for high resolution hyperspectral imaging , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[30]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[31]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[32]  Angshul Majumdar,et al.  Hyperspectral Unmixing in the Presence of Mixed Noise Using Joint-Sparsity and Total Variation , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[33]  Ricardo Augusto Borsoi,et al.  Coupled Tensor Decomposition for Hyperspectral and Multispectral Image Fusion with Variability , 2020, 2006.16968.

[34]  Jonathan Cheung-Wai Chan,et al.  Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network , 2018, Remote. Sens..

[35]  De LathauwerLieven Decompositions of a Higher-Order Tensor in Block TermsPart II: Definitions and Uniqueness , 2008 .

[36]  Seung-Jean Kim,et al.  Hyperspectral Image Unmixing via Alternating Projected Subgradients , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[37]  Wotao Yin,et al.  Improved Iteratively Reweighted Least Squares for Unconstrained Smoothed 퓁q Minimization , 2013, SIAM J. Numer. Anal..

[38]  B. Kunkel,et al.  ROSIS (Reflective Optics System Imaging Spectrometer) - A Candidate Instrument For Polar Platform Missions , 1988, Other Conferences.

[39]  José M. Bioucas-Dias,et al.  Semiblind Hyperspectral Unmixing in the Presence of Spectral Library Mismatches , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Wing-Kin Ma,et al.  Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach , 2018, IEEE Transactions on Signal Processing.

[41]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[42]  R. Bouldin Best approximation of a normal operator in the Schatten $p$-norm , 1980 .

[43]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[44]  Antonio J. Plaza,et al.  A Signal Processing Perspective on Hyperspectral Unmixing: Insights from Remote Sensing , 2014, IEEE Signal Processing Magazine.

[45]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[46]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[47]  Huikang Liu,et al.  Is There Any Recovery Guarantee with Coupled Structured Matrix Factorization for Hyperspectral Super-Resolution? , 2019, 2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).