Provable Model for Tensor Ring Completion

Tensor completion aims to recover a multi-dimensional array from its incomplete observations. The recently proposed tensor ring (TR) decomposition has powerful representation ability and shows promising performance in tensor completion, though they suffer from lack of theoretical guarantee. In this paper, we rigorously analyze the sample complexity of TR completion and find it also possesses the balance characteristic, which is consistent with the result of matrix completion. Inspired by this property we propose a nuclear norm minimization model and solve it by the alternating direction method of multipliers (ADMM). The experiments on synthetic data verify the theoretic analysis, and the numerical results of real-world data demonstrate that the proposed method gains great performance improvement in tensor completion compared with the state-of-the-art ones.

[1]  Xiaodong Wang,et al.  Tensor completion via adaptive sampling of tensor fibers: Application to efficient indoor RF fingerprinting , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Minh N. Do,et al.  Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train , 2016, IEEE Transactions on Image Processing.

[3]  Xianjun Shi,et al.  A Fixed Point Iterative Method for Low $n$-Rank Tensor Pursuit , 2013, IEEE Transactions on Signal Processing.

[4]  Canyi Lu,et al.  A Library of ADMM for Sparse and Low-rank Optimization version 1 . 1 , 2018 .

[5]  Lek-Heng Lim,et al.  Tensor network ranks , 2018, 1801.02662.

[6]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[7]  Jianting Cao,et al.  Higher-dimension Tensor Completion via Low-rank Tensor Ring Decomposition , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[8]  Chao Li,et al.  Rank Minimization on Tensor Ring: A New Paradigm in Scalable Tensor Decomposition and Completion , 2018, ArXiv.

[9]  J. Cirac,et al.  Algorithms for finite projected entangled pair states , 2014, 1405.3259.

[10]  Ce Zhu,et al.  Robust block tensor principal component analysis , 2020, Signal Process..

[11]  Jianting Cao,et al.  Completion of High Order Tensor Data with Missing Entries via Tensor-Train Decomposition , 2017, ICONIP.

[12]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[13]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[14]  Hong Cheng,et al.  Trace Norm Regularized CANDECOMP/PARAFAC Decomposition With Missing Data , 2015, IEEE Transactions on Cybernetics.

[15]  V. Aggarwal,et al.  Efficient Low Rank Tensor Ring Completion , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Hong Cheng,et al.  Generalized Higher Order Orthogonal Iteration for Tensor Learning and Decomposition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Vaneet Aggarwal,et al.  Tensor Completion by Alternating Minimization under the Tensor Train (TT) Model , 2016, ArXiv.

[18]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[20]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[21]  Qibin Zhao,et al.  Tensor-ring Nuclear Norm Minimization and Application for Visual : Data Completion , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Li Yan,et al.  An Adaptive Weighted Tensor Completion Method for the Recovery of Remote Sensing Images With Missing Data , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Johan A. K. Suykens,et al.  Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data , 2011, IEEE Signal Processing Letters.

[24]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[25]  Qibin Zhao,et al.  An Effective Tensor Completion Method Based on Multi-linear Tensor Ring Decomposition , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[26]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[27]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[28]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[29]  YUNING YANG,et al.  Rank-1 Tensor Properties with Applications to a Class of Tensor Optimization Problems , 2016, SIAM J. Optim..

[30]  Yuanyuan Liu,et al.  An Efficient Matrix Factorization Method for Tensor Completion , 2013, IEEE Signal Processing Letters.

[31]  Hisashi Kashima,et al.  Statistical Performance of Convex Tensor Decomposition , 2011, NIPS.

[32]  Zemin Zhang,et al.  Exact Tensor Completion Using t-SVD , 2015, IEEE Transactions on Signal Processing.

[33]  Andrzej Cichocki,et al.  Smooth PARAFAC Decomposition for Tensor Completion , 2015, IEEE Transactions on Signal Processing.

[34]  Jun Fang,et al.  An Iterative Reweighted Method for Tucker Decomposition of Incomplete Tensors , 2016, IEEE Transactions on Signal Processing.

[35]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[36]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Ce Zhu,et al.  Tensor rank learning in CP decomposition via convolutional neural network , 2019, Signal Process. Image Commun..

[38]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Masashi Sugiyama,et al.  Learning Efficient Tensor Representations with Ring-structured Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[41]  Chao Li,et al.  Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion , 2018, AAAI.

[42]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[43]  Yipeng Liu,et al.  Low-Rank Tensor Train Coefficient Array Estimation for Tensor-on-Tensor Regression , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Gonzalo Mateos,et al.  Rank Regularization and Bayesian Inference for Tensor Completion and Extrapolation , 2013, IEEE Transactions on Signal Processing.

[45]  Roman Orus,et al.  A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States , 2013, 1306.2164.

[46]  Ce Zhu,et al.  Improved Robust Tensor Principal Component Analysis via Low-Rank Core Matrix , 2018, IEEE Journal of Selected Topics in Signal Processing.

[47]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[48]  Massimiliano Pontil,et al.  A New Convex Relaxation for Tensor Completion , 2013, NIPS.

[49]  Wotao Yin,et al.  Parallel matrix factorization for low-rank tensor completion , 2013, ArXiv.

[50]  Ce Zhu,et al.  Low Rank Tensor Completion for Multiway Visual Data , 2018, 1805.03967.

[51]  Yipeng Liu,et al.  Low CP Rank and Tucker Rank Tensor Completion for Estimating Missing Components in Image Data , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[52]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[53]  Naoto Yokoya,et al.  Total-variation-regularized Tensor Ring Completion for Remote Sensing Image Reconstruction , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[55]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[56]  Lutz Warnke,et al.  On the Method of Typical Bounded Differences , 2012, Combinatorics, Probability and Computing.

[57]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[58]  Lars Grasedyck,et al.  Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format , 2015, SIAM J. Sci. Comput..

[59]  Ce Zhu,et al.  Image Completion Using Low Tensor Tree Rank and Total Variation Minimization , 2019, IEEE Transactions on Multimedia.

[60]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[61]  Lyman P. Hurd,et al.  Fractal image compression , 1993 .

[62]  Johan A. K. Suykens,et al.  A Rank-One Tensor Updating Algorithm for Tensor Completion , 2015, IEEE Signal Processing Letters.

[64]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[65]  José Ignacio Latorre,et al.  Image compression and entanglement , 2005, ArXiv.

[66]  Michael I. Jordan,et al.  A General Analysis of the Convergence of ADMM , 2015, ICML.