Self-Supervised Nonlinear Transform-Based Tensor Nuclear Norm for Multi-Dimensional Image Recovery

In this paper, we study multi-dimensional image recovery. Recently, transform-based tensor nuclear norm minimization methods are considered to capture low-rank tensor structures to recover third-order tensors in multi-dimensional image processing applications. The main characteristic of such methods is to perform the linear transform along the third mode of third-order tensors, and then compute tensor nuclear norm minimization on the transformed tensor so that the underlying low-rank tensors can be recovered. The main aim of this paper is to propose a nonlinear multilayer neural network to learn a nonlinear transform via the observed tensor data under self-supervision. The proposed network makes use of low-rank representation of transformed tensors and data-fitting between the observed tensor and the reconstructed tensor to construct the nonlinear transformation. Extensive experimental results on tensor completion, background subtraction, robust tensor completion, and snapshot compressive imaging are presented to demonstrate that the performance of the proposed method is better than that of state-of-the-art methods.

[1]  Qionghai Dai,et al.  Rank Minimization for Snapshot Compressive Imaging , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Lior Horesh,et al.  Tensor-Tensor Products for Optimal Representation and Compression , 2019, ArXiv.

[3]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Yunchao Wei,et al.  Low-Rank Tensor Completion With a New Tensor Nuclear Norm Induced by Invertible Linear Transforms , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Massimiliano Pontil,et al.  A New Convex Relaxation for Tensor Completion , 2013, NIPS.

[8]  B. R. Shivakumar,et al.  Performance évaluation of spectral angle mapper and spectral correlation mapper classifiers over multiple remote sensor data , 2017, 2017 Second International Conference on Electrical, Computer and Communication Technologies (ICECCT).

[9]  Chao Li,et al.  Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion , 2018, AAAI.

[10]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[11]  Ce Zhu,et al.  Robust Low-Rank Tensor Ring Completion , 2020, IEEE Transactions on Computational Imaging.

[12]  Jeffrey A. Fessler,et al.  Efficient Dynamic Parallel MRI Reconstruction for the Low-Rank Plus Sparse Model , 2019, IEEE Transactions on Computational Imaging.

[13]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[14]  Xi-Le Zhao,et al.  Dictionary Learning With Low-Rank Coding Coefficients for Tensor Completion , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Zemin Zhang,et al.  Exact Tensor Completion Using t-SVD , 2015, IEEE Transactions on Signal Processing.

[16]  Michael K. Ng,et al.  A Fast Algorithm for Cosine Transform Based Tensor Singular Value Decomposition , 2019, ArXiv.

[17]  Michael K. Ng,et al.  Framelet Representation of Tensor Nuclear Norm for Third-Order Tensor Completion , 2019, IEEE Transactions on Image Processing.

[18]  Hengyong Yu,et al.  Spectral CT Reconstruction—ASSIST: Aided by Self-Similarity in Image-Spectral Tensors , 2019, IEEE Transactions on Computational Imaging.

[19]  Wei Liu,et al.  Tensor Robust Principal Component Analysis with a New Tensor Nuclear Norm , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[21]  Sudhish N. George,et al.  DCT based weighted adaptive multi-linear data completion and denoising , 2018, Neurocomputing.

[22]  M. Kilmer,et al.  Tensor-Tensor Products with Invertible Linear Transforms , 2015 .

[23]  Michael K. Ng,et al.  Robust tensor completion using transformed tensor singular value decomposition , 2020, Numer. Linear Algebra Appl..

[24]  Jifeng Ning,et al.  Enhanced Nonconvex Low-Rank Approximation of Tensor Multi-Modes for Tensor Completion , 2020, IEEE Transactions on Computational Imaging.

[25]  G. Watson.  Characterization of the subdifferential of some matrix norms , 1992 .

[26]  Andrei Nicolae,et al.  PLU: The Piecewise Linear Unit Activation Function , 2018, ArXiv.

[27]  Xin Yuan,et al.  Generalized alternating projection based total variation minimization for compressive sensing , 2015, 2016 IEEE International Conference on Image Processing (ICIP).

[28]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[29]  Wensheng Zhang,et al.  The Twist Tensor Nuclear Norm for Video Completion , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Lei Zhang,et al.  Multispectral Images Denoising by Intrinsic Tensor Sparsity Regularization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[32]  Qi Xie,et al.  Kronecker-Basis-Representation Based Tensor Sparsity and Its Applications to Tensor Recovery , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Zheng Shou,et al.  Deep Tensor ADMM-Net for Snapshot Compressive Imaging , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[35]  Zhouchen Lin,et al.  Tensor Q-Rank: A New Data Dependent Tensor Rank , 2019, ArXiv.

[36]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[37]  Wei He,et al.  Guaranteed Matrix Completion Under Multiple Linear Transformations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Michael K. Ng,et al.  Patched-tube unitary transform for robust tensor completion , 2020, Pattern Recognit..

[39]  Volkan Cevher,et al.  A variational approach to stable principal component pursuit , 2014, UAI.

[40]  Maria Colomba Comes,et al.  ADMM-DIPTV: combining Total Variation and Deep Image Prior for image restoration , 2020, ArXiv.

[41]  M. Ng,et al.  Low Rank Tensor Completion With Poisson Observations , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Guihai Chen,et al.  Shearlet Enhanced Snapshot Compressive Imaging , 2020, IEEE Transactions on Image Processing.