Weighted Low-Rank Tensor Recovery for Hyperspectral Image Restoration

Hyperspectral imaging, providing abundant spatial and spectral information simultaneously, has attracted a lot of interest in recent years. Unfortunately, due to the hardware limitations, the hyperspectral image (HSI) is vulnerable to various degradations, such as noises (random noise), blurs (Gaussian and uniform blur), and downsampled (both spectral and spatial downsample), each corresponding to the HSI denoising, deblurring, and super-resolution tasks, respectively. Previous HSI restoration methods are designed for one specific task only. Besides, most of them start from the 1-D vector or 2-D matrix models and cannot fully exploit the structurally spectral–spatial correlation in 3-D HSI. To overcome these limitations, in this article, we propose a unified low-rank tensor recovery model for comprehensive HSI restoration tasks, in which nonlocal similarity within spectral–spatial cubic and spectral correlation are simultaneously captured by third-order tensors. Furthermore, to improve the capability and flexibility, we formulate it as a weighted low-rank tensor recovery (WLRTR) model by treating the singular values differently. We demonstrate the reweighed strategy, which has been extensively studied in the matrix, also greatly benefits the tensor modeling. We also consider the stripe noise in HSI as the sparse error by extending WLRTR to robust principal component analysis (WLRTR-RPCA). Extensive experiments demonstrate the proposed WLRTR models consistently outperform state-of-the-art methods in typical HSI low-level vision tasks, including denoising, destriping, deblurring, and super-resolution.

[1]  Yongqiang Zhao,et al.  Hyperspectral Image Denoising via Sparse Representation and Low-Rank Constraint , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Yi Ma,et al.  A non-negative sparse promoting algorithm for high resolution hyperspectral imaging , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Salah Bourennane,et al.  Denoising and Dimensionality Reduction Using Multilinear Tools for Hyperspectral Images , 2008, IEEE Geoscience and Remote Sensing Letters.

[5]  Lucien Wald,et al.  Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions , 2002 .

[6]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[7]  Naoto Yokoya,et al.  Hyperspectral Pansharpening: A Review , 2015, IEEE Geoscience and Remote Sensing Magazine.

[8]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[9]  Alexander F. H. Goetz,et al.  Determination of semi-arid landscape endmembers and seasonal trends using convex geometry spectral unmixing techniques , 1993 .

[10]  Gang Zhou,et al.  Hyperspectral Image Deconvolution with a Spectral-Spatial Total Variation Regularization , 2017 .

[11]  Deyu Meng,et al.  Robust Matrix Factorization with Unknown Noise , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Xinyuan Zhang,et al.  Nonlocal Low-Rank Tensor Factor Analysis for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision , 2016, International Journal of Computer Vision.

[14]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.

[15]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[16]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[17]  Qiang Zhang,et al.  Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Pierrick Coupé,et al.  Author manuscript, published in "Journal of Magnetic Resonance Imaging 2010;31(1):192-203" DOI: 10.1002/jmri.22003 Adaptive Non-Local Means Denoising of MR Images with Spatially Varying Noise Levels , 2010 .

[19]  Shen-En Qian,et al.  Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Sheng Zhong,et al.  HSI-DeNet: Hyperspectral Image Restoration via Convolutional Neural Network , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Naoto Yokoya,et al.  Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Saïd Ladjal,et al.  Toward Optimal Destriping of MODIS Data Using a Unidirectional Variational Model , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Liangpei Zhang,et al.  Hyperspectral Image Restoration Using Low-Rank Matrix Recovery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Shutao Li,et al.  Hyperspectral Image Super-Resolution via Non-local Sparse Tensor Factorization , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Guangming Shi,et al.  Low-Rank Tensor Approximation with Laplacian Scale Mixture Modeling for Multiframe Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Ajmal S. Mian,et al.  Sparse Spatio-spectral Representation for Hyperspectral Image Super-resolution , 2014, ECCV.

[27]  Liangpei Zhang,et al.  Stripe Noise Separation and Removal in Remote Sensing Images by Consideration of the Global Sparsity and Local Variational Properties , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Anand Rangarajan,et al.  Image Denoising Using the Higher Order Singular Value Decomposition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Liangpei Zhang,et al.  Hyperspectral Image Denoising Employing a Spectral–Spatial Adaptive Total Variation Model , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[31]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[32]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[33]  Naoto Yokoya,et al.  Non-Local Meets Global: An Integrated Paradigm for Hyperspectral Denoising , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Sheng Zhong,et al.  Hyper-Laplacian Regularized Unidirectional Low-Rank Tensor Recovery for Multispectral Image Denoising , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Ling Shao,et al.  Nonlocal Hierarchical Dictionary Learning Using Wavelets for Image Denoising , 2013, IEEE Transactions on Image Processing.

[36]  Yao Wang,et al.  Low-Rank Matrix Factorization under General Mixture Noise Distributions , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[38]  Michael K. Ng,et al.  Deblurring and Sparse Unmixing for Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[40]  Xin Huang,et al.  Hyperspectral image noise reduction based on rank-1 tensor decomposition , 2013 .

[41]  Naoto Yokoya,et al.  Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data , 2014, IEEE Transactions on Image Processing.

[42]  Qi Xie,et al.  Kronecker-Basis-Representation Based Tensor Sparsity and Its Applications to Tensor Recovery , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Guangming Shi,et al.  Hyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation , 2016, IEEE Transactions on Image Processing.

[44]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Karen O. Egiazarian,et al.  Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction , 2013, IEEE Transactions on Image Processing.

[46]  Yu-Chiang Frank Wang,et al.  Low-rank matrix recovery with structural incoherence for robust face recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Weisheng Dong,et al.  3D magnetic resonance image denoising using low-rank tensor approximation , 2016, Neurocomputing.

[48]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Ajmal S. Mian,et al.  Bayesian sparse representation for hyperspectral image super resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Wei Liu,et al.  Image Fusion with Local Spectral Consistency and Dynamic Gradient Sparsity , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Wei Wei,et al.  Cluster Sparsity Field for Hyperspectral Imagery Denoising , 2016, ECCV.

[52]  Caroline Fossati,et al.  Denoising of Hyperspectral Images Using the PARAFAC Model and Statistical Performance Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Shutao Li,et al.  Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Ping Zhong,et al.  Learning to Diversify Patch-Based Priors for Remote Sensing Image Restoration , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[55]  Masahiro Okuda,et al.  Local Spectral Component Decomposition for Multi-Channel Image Denoising , 2016, IEEE Transactions on Image Processing.

[56]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[57]  Qing Liu,et al.  A Truncated Nuclear Norm Regularization Method Based on Weighted Residual Error for Matrix Completion , 2016, IEEE Transactions on Image Processing.

[58]  Delu Zeng,et al.  Pan-Sharpening with a Hyper-Laplacian Penalty , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[59]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[61]  Gangyao Kuang,et al.  Hyperspectral Image Restoration Using Low-Rank Tensor Recovery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[62]  Liangpei Zhang,et al.  A Blind Restoration Method for Remote Sensing Images , 2012, IEEE Geoscience and Remote Sensing Letters.

[63]  Yi Chang,et al.  Anisotropic Spectral-Spatial Total Variation Model for Multispectral Remote Sensing Image Destriping , 2015, IEEE Transactions on Image Processing.

[64]  Yuan Xie,et al.  Hyperspectral Image Restoration via Iteratively Regularized Weighted Schatten $p$-Norm Minimization , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[65]  Stéphane Chrétien,et al.  Von Neumann's trace inequality for tensors , 2015 .

[66]  Ronald N. Bracewell,et al.  The Fourier Transform and Its Applications , 1966 .

[67]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[68]  Yi Yang,et al.  Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Deyu Meng,et al.  Denoising Hyperspectral Image With Non-i.i.d. Noise Structure , 2017, IEEE Transactions on Cybernetics.

[70]  L. Mirsky A trace inequality of John von Neumann , 1975 .

[71]  Yulong Wang,et al.  Graph-Regularized Low-Rank Representation for Destriping of Hyperspectral Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[72]  Hidekata Hontani,et al.  Missing Slice Recovery for Tensors Using a Low-Rank Model in Embedded Space , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[73]  Sheng Zhong,et al.  Remote Sensing Image Stripe Noise Removal: From Image Decomposition Perspective , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[74]  Xuelong Li,et al.  Spectral-Spatial Constraint Hyperspectral Image Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[75]  Jean-Yves Tourneret,et al.  Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[76]  S. Osher,et al.  Image restoration: Total variation, wavelet frames, and beyond , 2012 .

[77]  Yan Liu,et al.  Weighted Schatten $p$ -Norm Minimization for Image Denoising and Background Subtraction , 2015, IEEE Transactions on Image Processing.

[78]  Bo Du,et al.  A Discriminative Metric Learning Based Anomaly Detection Method , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[79]  Liangpei Zhang,et al.  A MAP-Based Algorithm for Destriping and Inpainting of Remotely Sensed Images , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[80]  Konrad Schindler,et al.  Hyperspectral Super-Resolution by Coupled Spectral Unmixing , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[81]  Pierre Weiss,et al.  Variational Algorithms to Remove Stationary Noise: Applications to Microscopy Imaging , 2012, IEEE Transactions on Image Processing.

[82]  Salah Bourennane,et al.  Noise Removal From Hyperspectral Images by Multidimensional Filtering , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[83]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[84]  Frédo Durand,et al.  Understanding and evaluating blind deconvolution algorithms , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[85]  Yoichi Sato,et al.  Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Restoration , 2017, International Journal of Computer Vision.

[86]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[87]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[88]  Hervé Carfantan,et al.  Statistical Linear Destriping of Satellite-Based Pushbroom-Type Images , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[89]  Lei Zhang,et al.  Multispectral Images Denoising by Intrinsic Tensor Sparsity Regularization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[90]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[91]  Qi Xie,et al.  Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[93]  Charles Soussen,et al.  Fast Positive Deconvolution of Hyperspectral Images , 2013, IEEE Transactions on Image Processing.

[94]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[95]  Jiangjun Peng,et al.  Hyperspectral Image Restoration Via Total Variation Regularized Low-Rank Tensor Decomposition , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[96]  Liangpei Zhang,et al.  Total-Variation-Regularized Low-Rank Matrix Factorization for Hyperspectral Image Restoration , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[97]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[98]  Jun Zhou,et al.  Multitask Sparse Nonnegative Matrix Factorization for Joint Spectral–Spatial Hyperspectral Imagery Denoising , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[99]  Lei Zheng,et al.  Image Noise Level Estimation by Principal Component Analysis , 2013, IEEE Transactions on Image Processing.

[100]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[101]  Emanuela Lupo,et al.  Hyperspectral Image Analysis in Environmental Monitoring: Setup of a New Tunable Filter Platform , 2013 .