Hyperspectral Image Restoration Combining Intrinsic Image Characterization With Robust Noise Modeling

In hyperspectral image (HSI) processing, a fundamental issue is to restore HSI data from various degradations such as noise corruption and information missing. However, most existing methods more or less ignore the abundant prior knowledge on HSIs and the embedded noise, leading to suboptimal performance in practice. In this article, we propose a novel HSI restoration method by fully considering the intrinsic image structures and the complex noise characteristics. For HSIs, the global correlation is captured by the Kronecker-basis-representation-based tensor low-rankness measure, which integrates the insights delivered by both CP and Tucker decompositions; the local regularity is depicted by a plug-and-play spatial-spectral convolutional neural network with strong fitting ability to complex image features. For realistic noise, its statistical characteristics are encoded by a nonidentical and nonindependent distributed mixture of Gaussians distribution with flexible fitting capability. Then, we incorporate these image and noise priors into a probabilistic model based on the maximum a posteriori principle, and develop a solving scheme by combining expectation-maximization and alternating direction method of multipliers. Extensive experimental results on both simulated and real scenarios demonstrate the effectiveness of the proposed method and its superiority over the compared state-of-the- arts.

[1]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[2]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[3]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[4]  Zhijun Zhang,et al.  Weighted Low-Rank Tensor Recovery for Hyperspectral Image Restoration , 2017, IEEE Transactions on Cybernetics.

[5]  Yongqiang Zhao,et al.  Hyperspectral Image Denoising via Sparse Representation and Low-Rank Constraint , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[6]  L. Mirsky A trace inequality of John von Neumann , 1975 .

[7]  Deyu Meng,et al.  Robust Matrix Factorization with Unknown Noise , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Yandong Tang,et al.  A Generalized Model for Robust Tensor Factorization With Noise Modeling by Mixture of Gaussians , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[9]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[10]  Yipeng Liu,et al.  Low CP Rank and Tucker Rank Tensor Completion for Estimating Missing Components in Image Data , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Sheng Zhong,et al.  Hyperspectral Image Restoration: Where Does the Low-Rank Property Exist , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Deyu Meng,et al.  Robust Low-Rank Matrix Factorization Under General Mixture Noise Distributions , 2016, IEEE Transactions on Image Processing.

[13]  Shen-En Qian,et al.  Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Sheng Zhong,et al.  HSI-DeNet: Hyperspectral Image Restoration via Convolutional Neural Network , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Johannes R. Sveinsson,et al.  Hyperspectral image denoising using 3D wavelets , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[16]  Jocelyn Chanussot,et al.  Noise Reduction in Hyperspectral Imagery: Overview and Application , 2018, Remote. Sens..

[17]  Yan Yan,et al.  $L_{1}$ -Norm Low-Rank Matrix Factorization by Variational Bayesian Method , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Liangpei Zhang,et al.  Hyperspectral Image Denoising Using Local Low-Rank Matrix Recovery and Global Spatial–Spectral Total Variation , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Jie Lin,et al.  A Tensor Subspace Representation-Based Method for Hyperspectral Image Denoising , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Salah Bourennane,et al.  Denoising and Dimensionality Reduction Using Multilinear Tools for Hyperspectral Images , 2008, IEEE Geoscience and Remote Sensing Letters.

[21]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Deyu Meng,et al.  Remote Sensing Image Denoising via Low-Rank Tensor Approximation and Robust Noise Modeling , 2020, Remote. Sens..

[23]  Naoto Yokoya,et al.  Remote Sensing Image Reconstruction Using Tensor Ring Completion and Total Variation , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[24]  V. Maz'ya,et al.  On approximate approximations using Gaussian kernels , 1996 .

[25]  H. Sebastian Seung,et al.  Natural Image Denoising with Convolutional Networks , 2008, NIPS.

[26]  Naoto Yokoya,et al.  Non-Local Meets Global: An Integrated Paradigm for Hyperspectral Denoising , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Naoto Yokoya,et al.  Nonlocal Tensor-Ring Decomposition for Hyperspectral Image Denoising , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Deyu Meng,et al.  Denoising Hyperspectral Image With Non-i.i.d. Noise Structure , 2017, IEEE Transactions on Cybernetics.

[29]  Pedram Ghamisi,et al.  Automatic Hyperspectral Image Restoration Using Sparse and Low-Rank Modeling , 2017, IEEE Geoscience and Remote Sensing Letters.

[30]  Qi Xie,et al.  Kronecker-Basis-Representation Based Tensor Sparsity and Its Applications to Tensor Recovery , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[33]  Pierre Vandergheynst,et al.  Joint trace/TV norm minimization: A new efficient approach for spectral compressive imaging , 2012, 2012 19th IEEE International Conference on Image Processing.

[34]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[35]  Liangpei Zhang,et al.  Hyperspectral Image Restoration Using Low-Rank Matrix Recovery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Xiaohan Chen,et al.  Plug-and-Play Methods Provably Converge with Properly Trained Denoisers , 2019, ICML.

[37]  Yao Wang,et al.  Deep Plug-and-play Prior for Low-rank Tensor Completion , 2019, Neurocomputing.

[38]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[39]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[40]  Caroline Fossati,et al.  Denoising of Hyperspectral Images Using the PARAFAC Model and Statistical Performance Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Gangyao Kuang,et al.  Hyperspectral Image Restoration Using Low-Rank Tensor Recovery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[43]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[44]  Yee Leung,et al.  Robust Multiview Subspace Learning With Nonindependently and Nonidentically Distributed Complex Noise , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Xiaoming Tao,et al.  Hyperspectral Image Denoising via Matrix Factorization and Deep Prior Regularization , 2020, IEEE Transactions on Image Processing.

[46]  Lucien Wald,et al.  Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions , 2002 .

[47]  Naoto Yokoya,et al.  Nonlocal Tensor Completion for Multitemporal Remotely Sensed Images’ Inpainting , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Jiangjun Peng,et al.  Hyperspectral Image Restoration Via Total Variation Regularized Low-Rank Tensor Decomposition , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[49]  Xinghua Li,et al.  Cloud removal in remote sensing images using nonnegative matrix factorization and error correction , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[50]  João Paulo Papa,et al.  Projections Onto Convex Sets through Particle Swarm Optimization and its application for remote sensing image restoration , 2010, Pattern Recognit. Lett..

[51]  Jon Atli Benediktsson,et al.  Hyperspectral Mixed Gaussian and Sparse Noise Reduction , 2020, IEEE Geoscience and Remote Sensing Letters.

[52]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[53]  Minh N. Do,et al.  Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train , 2016, IEEE Transactions on Image Processing.

[54]  Gang Yang,et al.  Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[55]  Alexander F. H. Goetz,et al.  Determination of semi-arid landscape endmembers and seasonal trends using convex geometry spectral unmixing techniques , 1993 .

[56]  Lei Zhang,et al.  Robust Principal Component Analysis with Complex Noise , 2014, ICML.

[57]  Angshul Majumdar,et al.  Hyperspectral Image Denoising Using Spatio-Spectral Total Variation , 2016, IEEE Geoscience and Remote Sensing Letters.

[58]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[59]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[60]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[61]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Ting-Zhu Huang,et al.  Remote sensing images destriping using unidirectional hybrid total variation and nonconvex low-rank regularization , 2020, J. Comput. Appl. Math..

[63]  Brendt Wohlberg,et al.  Plug-and-Play priors for model based reconstruction , 2013, 2013 IEEE Global Conference on Signal and Information Processing.