Hyperspectral Images Super-Resolution via Learning High-Order Coupled Tensor Ring Representation

Hyperspectral image (HSI) super-resolution is a hot topic in remote sensing and computer vision. Recently, tensor analysis has been proven to be an efficient technology for HSI image processing. However, the existing tensor-based methods of HSI super-resolution are not able to capture the high-order correlations in HSI. In this article, we propose to learn a high-order coupled tensor ring (TR) representation for HSI super-resolution. The proposed method first tensorizes the HSI to be estimated into a high-order tensor in which multiscale spatial structures and the original spectral structure are represented. Then, a coupled TR representation model is proposed to fuse the low-resolution HSI (LR-HSI) and high-resolution multispectral image (HR-MSI). In the proposed model, some latent core tensors in TR of the LR-HSI and the HR-MSI are shared, and we use the relationship between the spectral core tensors to reconstruct the HSI. In addition, the graph-Laplacian regularization is introduced to the spectral core tensors to preserve the spectral information. To enhance the robustness of the proposed model, Frobenius norm regularizations are introduced to the other core tensors. Experimental results on both synthetic and real data sets show that the proposed method achieves the state-of-the-art super-resolution performance.

[1]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[2]  Antonio J. Plaza,et al.  Anomaly Detection in Hyperspectral Images Based on Low-Rank and Sparse Representation , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  V. Aggarwal,et al.  Efficient Low Rank Tensor Ring Completion , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Guangming Shi,et al.  Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation , 2016, IEEE Transactions on Image Processing.

[6]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[7]  Shutao Li,et al.  Hyperspectral Image Super-Resolution via Non-local Sparse Tensor Factorization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Johan A. K. Suykens,et al.  Parallelized Tensor Train Learning of Polynomial Classifiers , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Yandong Tang,et al.  A Generalized Model for Robust Tensor Factorization With Noise Modeling by Mixture of Gaussians , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Le Sun,et al.  Low Rank Component Induced Spatial-Spectral Kernel Method for Hyperspectral Image Classification , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Yuan Xie,et al.  Hyperspectral Image Restoration via Iteratively Regularized Weighted Schatten $p$-Norm Minimization , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[13]  Boxin Shi,et al.  Self-Similarity Constrained Sparse Representation for Hyperspectral Image Super-Resolution , 2018, IEEE Transactions on Image Processing.

[14]  Naoto Yokoya,et al.  An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing , 2018, IEEE Transactions on Image Processing.

[15]  Qi Wang,et al.  Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Xiangtao Zheng,et al.  Joint Dictionary Learning for Multispectral Change Detection , 2017, IEEE Transactions on Cybernetics.

[17]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[18]  Huachun Tan,et al.  A Fused CP Factorization Method for Incomplete Tensors , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[20]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Shutao Li,et al.  Deep Hyperspectral Image Sharpening , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Yunming Ye,et al.  MR-NTD: Manifold Regularization Nonnegative Tucker Decomposition for Tensor Data Dimension Reduction and Representation , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[25]  Yan Liu,et al.  Weighted Schatten $p$ -Norm Minimization for Image Denoising and Background Subtraction , 2015, IEEE Transactions on Image Processing.

[26]  Zhetao Li,et al.  Three-Dimensional Attention-Based Deep Ranking Model for Video Highlight Detection , 2018, IEEE Transactions on Multimedia.

[27]  Yong Luo,et al.  Tensor Canonical Correlation Analysis for Multi-View Dimension Reduction , 2015, IEEE Transactions on Knowledge and Data Engineering.

[28]  Xiao Xiang Zhu,et al.  Object-Based Multipass InSAR via Robust Low-Rank Tensor Decomposition , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Shutao Li,et al.  Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Shuyuan Yang,et al.  Multispectral and Hyperspectral Image Fusion Based on Group Spectral Embedding and Low-Rank Factorization , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[32]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[34]  Jun Li,et al.  Feature-Driven Active Learning for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Yuan Xie,et al.  On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization , 2016, International Journal of Computer Vision.

[36]  Shutao Li,et al.  Super-resolution of hyperspectral image via superpixel-based sparse representation , 2018, Neurocomputing.

[37]  Naoto Yokoya,et al.  Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data , 2014, IEEE Transactions on Image Processing.

[38]  Shutao Li,et al.  Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization , 2018, IEEE Transactions on Image Processing.

[39]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Yoichi Sato,et al.  Hyperspectral Image Super-Resolution With a Mosaic RGB Image , 2018, IEEE Transactions on Image Processing.

[41]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Chao Li,et al.  Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion , 2018, AAAI.

[43]  Lei Zhang,et al.  Exploiting Clustering Manifold Structure for Hyperspectral Imagery Super-Resolution , 2018, IEEE Transactions on Image Processing.

[44]  Liqing Zhang,et al.  Bayesian Robust Tensor Factorization for Incomplete Multiway Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Masashi Sugiyama,et al.  Learning Efficient Tensor Representations with Ring-structured Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[47]  Liqing Zhang,et al.  Tensor Ring Decomposition , 2016, ArXiv.

[48]  Masashi Sugiyama,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives , 2017, Found. Trends Mach. Learn..

[49]  Qinghua Hu,et al.  Generalized Latent Multi-View Subspace Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[51]  Liang Xiao,et al.  Supervised Spectral–Spatial Hyperspectral Image Classification With Weighted Markov Random Fields , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Nikos D. Sidiropoulos,et al.  Hyperspectral Super-Resolution Via Coupled Tensor Factorization: Identifiability and Algorithms , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[55]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[56]  Nikos D. Sidiropoulos,et al.  Hyperspectral Super-Resolution: Combining Low Rank Tensor and Matrix Structure , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[57]  Antonio J. Plaza,et al.  Hyperspectral Unmixing Overview: Geometrical, Statistical, and Sparse Regression-Based Approaches , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[58]  Ajmal S. Mian,et al.  Bayesian sparse representation for hyperspectral image super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Menas Kafatos,et al.  Wavelet-based hyperspectral and multispectral image fusion , 2001, SPIE Defense + Commercial Sensing.