Dictionary Learning With Low-Rank Coding Coefficients for Tensor Completion

In this paper, we propose a novel tensor learning and coding model for third-order data completion. Our model is to learn a data-adaptive dictionary from the given observations, and determine the coding coefficients of third-order tensor tubes. In the completion process, we minimize the low-rankness of each tensor slice containing the coding coefficients. By comparison with the traditional pre-defined transform basis, the advantages of the proposed model are that (i) the dictionary can be learned based on the given data observations so that the basis can be more adaptively and accurately constructed, and (ii) the low-rankness of the coding coefficients can allow the linear combination of dictionary features more effectively. Also we develop a multi-block proximal alternating minimization algorithm for solving such tensor learning and coding model, and show that the sequence generated by the algorithm can globally converge to a critical point. Extensive experimental results for real data sets such as videos, hyperspectral images, and traffic data are reported to demonstrate these advantages and show the performance of the proposed tensor learning and coding method is significantly better than the other tensor completion methods in terms of several evaluation metrics.

[1]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Qiang Jiang,et al.  Robust Low-Tubal-Rank Tensor Completion via Convex Optimization , 2019, IJCAI.

[3]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[4]  Chao Li,et al.  Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion , 2018, AAAI.

[5]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[6]  Minh N. Do,et al.  Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train , 2016, IEEE Transactions on Image Processing.

[7]  José M. Bioucas-Dias,et al.  Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[10]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[11]  Michael K. Ng,et al.  A Fast Algorithm for Cosine Transform Based Tensor Singular Value Decomposition , 2019, ArXiv.

[12]  M. Kilmer,et al.  Factorization strategies for third-order tensors , 2011 .

[13]  Zemin Zhang,et al.  Exact Tensor Completion Using t-SVD , 2015, IEEE Transactions on Signal Processing.

[14]  Christopher Rasmussen,et al.  Spatiotemporal Inpainting for Recovering Texture Maps of Occluded Building Facades , 2007, IEEE Transactions on Image Processing.

[15]  Liqing Zhang,et al.  Bayesian Robust Tensor Factorization for Incomplete Multiway Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[17]  Shutao Li,et al.  Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Yandong Tang,et al.  A Generalized Model for Robust Tensor Factorization With Noise Modeling by Mixture of Gaussians , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Michael K. Ng,et al.  Robust tensor completion using transformed tensor singular value decomposition , 2020, Numer. Linear Algebra Appl..

[20]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[21]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  M. Kilmer,et al.  Tensor-Tensor Products with Invertible Linear Transforms , 2015 .

[23]  Yi Yang,et al.  Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[25]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[26]  Guido Schuster,et al.  High spatio-temporal resolution video with compressed sensing. , 2015, Optics express.

[27]  Michael K. Ng,et al.  Framelet Representation of Tensor Nuclear Norm for Third-Order Tensor Completion , 2019, IEEE Transactions on Image Processing.

[28]  Yunchao Wei,et al.  Low-Rank Tensor Completion With a New Tensor Nuclear Norm Induced by Invertible Linear Transforms , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Karen S. Braman Third-Order Tensors as Linear Operators on a Space of Matrices , 2010 .

[30]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[31]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[32]  Ce Zhu,et al.  Low Rank Tensor Completion for Multiway Visual Data , 2018, 1805.03967.

[33]  Xiongjun Zhang,et al.  A Nonconvex Relaxation Approach to Low-Rank Tensor Completion , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[34]  James Caverlee,et al.  Tensor Completion Algorithms in Big Data Analytics , 2017, ACM Trans. Knowl. Discov. Data.

[35]  Tomer Michaeli,et al.  Multi-scale Weighted Nuclear Norm Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[37]  Alexander F. H. Goetz,et al.  Determination of semi-arid landscape endmembers and seasonal trends using convex geometry spectral unmixing techniques , 1993 .

[38]  Chao Li,et al.  Low Tensor-Ring Rank Completion by Parallel Matrix Factorization , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Yipeng Liu,et al.  Low-Rank Tensor Train Coefficient Array Estimation for Tensor-on-Tensor Regression , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[40]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[41]  Liqing Zhang,et al.  Tensor Ring Decomposition , 2016, ArXiv.

[42]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[43]  M. Sabarimalai Manikandan,et al.  Adaptive MRI image denoising using total-variation and local noise estimation , 2012, IEEE-International Conference On Advances In Engineering, Science And Management (ICAESM -2012).

[44]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[45]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[46]  Hristo S. Sendov,et al.  Nonsmooth Analysis of Singular Values. Part II: Applications , 2005 .

[47]  Zuowei Shen,et al.  Dictionary Learning for Sparse Coding: Algorithms and Convergence Analysis , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[49]  Nikos Komodakis,et al.  Image Completion Using Global Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).