Hyperspectral Image Superresolution Using Global Gradient Sparse and Nonlocal Low-Rank Tensor Decomposition With Hyper-Laplacian Prior

This article presents a novel global gradient sparse and nonlocal low-rank tensor decomposition model with a hyper-Laplacian prior for hyperspectral image (HSI) superresolution to produce a high-resolution HSI (HR-HSI) by fusing a low-resolution HSI (LR-HSI) with an HR multispectral image (HR-MSI). Inspired by the investigated hyper-Laplacian distribution of the gradients of the difference images between the upsampled LR-HSI and latent HR-HSI, we formulate the relationship between these two datasets as a $\ell _{p}$ $(0 < p < 1)$-norm term to enforce spectral preservation. Then, the relationship between the HR-MSI and latent HR-HSI is built using a tensor-based fidelity term to recover the spatial details. To effectively capture the high spatio-spectral-nonlocal similarities of the latent HR-HSI, we design a novel nonlocal low-rank Tucker decomposition to model the 3-D regular tensors constructed from the grouped nonlocal similar HR-HSI cubes. The global spatial-spectral total variation regularization is then adopted to ensure the global spatial piecewise smoothness and spectral consistency of the reconstructed HR-HSI from nonlocal low-rank cubes. Finally, an alternating direction method of multipliers-based algorithm is designed to efficiently solve the optimization problem. Experiments on both the synthetic and real datasets collected by different sensors show the effectiveness of the proposed method, from visual and quantitative assessments.

[1]  Jiangjun Peng,et al.  Hyperspectral Image Restoration Via Total Variation Regularized Low-Rank Tensor Decomposition , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[3]  Shuyuan Yang,et al.  Spatial–Spectral-Graph-Regularized Low-Rank Tensor Decomposition for Multispectral and Hyperspectral Image Fusion , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Simon J. Godsill,et al.  R-FUSE: Robust Fast Fusion of Multiband Images Based on Solving a Sylvester Equation , 2016, IEEE Signal Processing Letters.

[5]  Y. She,et al.  Thresholding-based iterative selection procedures for model selection and shrinkage , 2008, 0812.5061.

[6]  Gangyao Kuang,et al.  Spatial–Spectral Total Variation Regularized Low-Rank Tensor Decomposition for Hyperspectral Image Denoising , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Yiyuan She,et al.  Thresholding-based Iterative Selection Procedures for Generalized Linear Models , 2009, 0911.5460.

[8]  Michael Elad,et al.  Image Processing Using Smooth Ordering of its Patches , 2012, IEEE Transactions on Image Processing.

[9]  S. Baronti,et al.  Remote Sensing Image Fusion , 2015 .

[10]  Shutao Li,et al.  Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Jocelyn Chanussot,et al.  MultiResolution Analysis and Component Substitution techniques for hyperspectral Pansharpening , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[12]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Todd Wittman,et al.  A variational approach to hyperspectral image fusion , 2009, Defense + Commercial Sensing.

[14]  Jocelyn Chanussot,et al.  Nonlocal Patch Tensor Sparse Representation for Hyperspectral Image Super-Resolution , 2019, IEEE Transactions on Image Processing.

[15]  Olivier Berné,et al.  Non-negative matrix factorization pansharpening of hyperspectral data: An application to mid-infrared astronomy , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[16]  Liangpei Zhang,et al.  Hyperspectral Image Denoising Using Local Low-Rank Matrix Recovery and Global Spatial–Spectral Total Variation , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Yi Ma,et al.  A non-negative sparse promoting algorithm for high resolution hyperspectral imaging , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[19]  Liangpei Zhang,et al.  A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpening , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[20]  Shutao Li,et al.  A New Pan-Sharpening Method Using a Compressed Sensing Technique , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Shutao Li,et al.  Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization , 2018, IEEE Transactions on Image Processing.

[22]  Junfeng Yang,et al.  Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization , 2012, Math. Comput..

[23]  Jun Liu,et al.  Nonnegative and Nonlocal Sparse Tensor Factorization-Based Hyperspectral Image Super-Resolution , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Xiangtao Zheng,et al.  Hyperspectral Image Superresolution by Transfer Learning , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[25]  Hankui K. Zhang,et al.  A New Look at Image Fusion Methods from a Bayesian Perspective , 2015, Remote. Sens..

[26]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[27]  Naoto Yokoya,et al.  Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature , 2017, IEEE Geoscience and Remote Sensing Magazine.

[28]  Naoto Yokoya,et al.  Nonlocal Tensor-Ring Decomposition for Hyperspectral Image Denoising , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Hairong Qi,et al.  Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Jocelyn Chanussot,et al.  A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Bin Wang,et al.  Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods , 2014, IEEE Geoscience and Remote Sensing Letters.

[32]  S. Baronti,et al.  Twenty-Five Years of Pansharpening: A Critical Review and New Developments , 2012 .

[33]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[34]  Shutao Li,et al.  Super-resolution of hyperspectral image via superpixel-based sparse representation , 2018, Neurocomputing.

[35]  Xue-Cheng Tai,et al.  The fusion of panchromatic and multispectral remote sensing images via tensor-based sparse modeling and hyper-Laplacian prior , 2019, Inf. Fusion.

[36]  Jocelyn Chanussot,et al.  Synthesis of Multispectral Images to High Spatial Resolution: A Critical Review of Fusion Methods Based on Remote Sensing Physics , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Wei He,et al.  Hyperspectral Image Denoising With Total Variation Regularization and Nonlocal Low-Rank Tensor Decomposition , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Gary A. Shaw,et al.  Spectral Imaging for Remote Sensing , 2003 .

[39]  Zhijun Zhang,et al.  Weighted Low-Rank Tensor Recovery for Hyperspectral Image Restoration , 2017, IEEE Transactions on Cybernetics.

[40]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[41]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[42]  Shutao Li,et al.  Deep Hyperspectral Image Sharpening , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Liangpei Zhang,et al.  Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: Practical discussion and challenges , 2019, Inf. Fusion.

[45]  Guangming Shi,et al.  Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation , 2016, IEEE Transactions on Image Processing.

[46]  J. Boardman,et al.  Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm , 1992 .

[47]  Jean-Yves Tourneret,et al.  Bayesian Fusion of Multi-Band Images , 2013, IEEE Journal of Selected Topics in Signal Processing.

[48]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[49]  Ricardo Augusto Borsoi,et al.  Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability , 2018, IEEE Transactions on Image Processing.

[50]  Jean-Yves Tourneret,et al.  Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation , 2015, IEEE Transactions on Image Processing.

[51]  Simon J. Godsill,et al.  Multiband Image Fusion Based on Spectral Unmixing , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[52]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[53]  Andrzej Cichocki,et al.  Total Variation Regularized Tensor RPCA for Background Subtraction From Compressive Measurements , 2015, IEEE Transactions on Image Processing.

[54]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[55]  David Zhang,et al.  A Generalized Iterated Shrinkage Algorithm for Non-convex Sparse Coding , 2013, 2013 IEEE International Conference on Computer Vision.

[56]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Nebiye Musaoglu,et al.  Merging hyperspectral and panchromatic image data: qualitative and quantitative analysis , 2009 .

[58]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[59]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Lei Zhang,et al.  Multispectral Images Denoising by Intrinsic Tensor Sparsity Regularization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  William J. Emery,et al.  An Integrated Spatio-Spectral–Temporal Sparse Representation Method for Fusing Remote-Sensing Images With Different Resolutions , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[62]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[63]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Shutao Li,et al.  Hyperspectral Image Super-Resolution via Non-local Sparse Tensor Factorization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).