Fast and Accurate Tensor Completion with Tensor Trains: A System Identification Approach

We propose a novel tensor completion approach by equating it to a system identification task. The key is to regard the coordinates and values of the known entries as inputs and outputs, respectively. By assuming a tensor train format initialized with low-rank tensor cores, the latter are iteratively identified via a simple alternating linear scheme to reduce residuals. Experiments verify the superiority of the proposed scheme in terms of both speed and accuracy, where a speedup of up to 23$\times$ is observed compared to state-of-the-art tensor completion methods at a similar accuracy.

[1]  Ngai Wong,et al.  A Tensor Network Kalman filter with an application in recursive MIMO Volterra system identification , 2017, Autom..

[2]  Arieh Iserles,et al.  On the Foundations of Computational Mathematics , 2001 .

[3]  J. Coyle.  Inverse Problems , 2004 .

[4]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[5]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[7]  Roman Orus,et al.  A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States , 2013, 1306.2164.

[8]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Lars Grasedyck,et al.  Alternating Least Squares Tensor Completion in The TT-Format , 2015, 1509.00311.

[10]  Lars Grasedyck,et al.  Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format , 2015, SIAM J. Sci. Comput..

[11]  Tamara G. Kolda,et al.  Higher-order Web link analysis using multilinear algebra , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[12]  Martin Burger,et al.  Inverse problems in imaging , 2013 .

[13]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[14]  Nicholas I. M. Gould,et al.  SIAM Journal on Optimization , 2012 .

[15]  V. Aggarwal,et al.  Efficient Low Rank Tensor Ring Completion , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Xuanjing Huang,et al.  Learning Context-Sensitive Word Embeddings with Neural Tensor Skip-Gram Model , 2015, IJCAI.

[17]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[18]  Nuria Oliver,et al.  Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering , 2010, RecSys '10.

[19]  Taiji Suzuki,et al.  Convergence rate of Bayesian tensor estimator and its minimax optimality , 2015, ICML.

[20]  Misha Elena Kilmer,et al.  Third-Order Tensors as Operators on Matrices: A Theoretical and Computational Framework with Applications in Imaging , 2013, SIAM J. Matrix Anal. Appl..

[21]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[22]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[23]  Guillermo Sapiro,et al.  Video Inpainting Under Constrained Camera Motion , 2007, IEEE Transactions on Image Processing.

[24]  Ivan V. Oseledets,et al.  Approximation of 2d˟2d Matrices Using Tensor Decomposition , 2010, SIAM J. Matrix Anal. Appl..

[25]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[26]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[27]  U. Schollwoeck.  The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[28]  Zhen Zhang,et al.  Improved Bounded Matrix Completion for Large-Scale Recommender Systems , 2017, IJCAI.

[29]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[30]  Congfu Xu,et al.  Tensor Completion with Side Information: A Riemannian Manifold Approach , 2017, IJCAI.

[31]  Eric L. Miller,et al.  Tensor-Based Formulation and Nuclear Norm Regularization for Multienergy Computed Tomography , 2013, IEEE Transactions on Image Processing.

[32]  Ngai Wong,et al.  Tensor Network alternating linear scheme for MIMO Volterra system identification , 2016, Autom..

[33]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Massimiliano Pontil,et al.  Multilinear Multitask Learning , 2013, ICML.

[35]  Minh N. Do,et al.  Efficient tensor completion: Low-rank tensor train , 2016, ArXiv.

[36]  Johan A. K. Suykens,et al.  Learning with tensors: a framework based on convex optimization and spectral regularization , 2014, Machine Learning.

[37]  Guillermo Sapiro,et al.  Dictionary Learning for Noisy and Incomplete Hyperspectral Images , 2012, SIAM J. Imaging Sci..

[38]  Yunming Ye,et al.  Low-Rank Tensor Completion with Total Variation for Visual Data Inpainting , 2017, AAAI.

[39]  Bin Ran,et al.  Tensor completion via a multi-linear low-n-rank factorization model , 2014, Neurocomputing.

[40]  André Uschmajew,et al.  On Local Convergence of Alternating Schemes for Optimization of Convex Problems in the Tensor Train Format , 2013, SIAM J. Numer. Anal..

[41]  Gang Liu,et al.  Tensor completion using total variation and low-rank matrix factorization , 2016, Inf. Sci..

[42]  Yudong Chen,et al.  Incoherence-Optimal Matrix Completion , 2013, IEEE Transactions on Information Theory.

[43]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[44]  Takanori Maehara,et al.  On Tensor Train Rank Minimization : Statistical Efficiency and Scalable Algorithm , 2017, NIPS.

[45]  Wotao Yin,et al.  Parallel matrix factorization for low-rank tensor completion , 2013, ArXiv.

[46]  Shay B. Cohen,et al.  Tensor Decomposition for Fast Parsing with Latent-Variable PCFGs , 2012, NIPS.

[47]  Nadia Kreimer,et al.  A tensor higher-order singular value decomposition for prestack seismic data noise reduction and interpolation , 2012 .

[48]  René Vidal,et al.  A Structured Sparse Plus Structured Low-Rank Framework for Subspace Clustering and Completion , 2016, IEEE Transactions on Signal Processing.

[49]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[50]  Kim Batselier,et al.  Tensor Computation: A New Framework for High-Dimensional Problems in EDA , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[51]  Minh N. Do,et al.  Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train , 2016, IEEE Transactions on Image Processing.

[52]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Masashi Sugiyama,et al.  Learning Efficient Tensor Representations with Ring-structured Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).