HOSVD-Based Algorithm for Weighted Tensor Completion

Matrix completion, the problem of completing missing entries in a data matrix with low-dimensional structure (such as rank), has seen many fruitful approaches and analyses. Tensor completion is the tensor analog that attempts to impute missing tensor entries from similar low-rank type assumptions. In this paper, we study the tensor completion problem when the sampling pattern is deterministic and possibly non-uniform. We first propose an efficient weighted Higher Order Singular Value Decomposition (HOSVD) algorithm for the recovery of the underlying low-rank tensor from noisy observations and then derive the error bounds under a properly weighted metric. Additionally, the efficiency and accuracy of our algorithm are both tested using synthetic and real datasets in numerical simulations.

[1]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[2]  Donald Goldfarb,et al.  Robust Low-Rank Tensor Recovery: Models and Algorithms , 2013, SIAM J. Matrix Anal. Appl..

[3]  Jiangjun Peng,et al.  Hyperspectral Image Restoration Via Total Variation Regularized Low-Rank Tensor Decomposition , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[5]  Xiaodong Wang,et al.  On Deterministic Sampling Patterns for Robust Low-Rank Matrix Completion , 2017, IEEE Signal Processing Letters.

[6]  Mark A. Iwen,et al.  Extension of PCA to Higher Order Data Structures: An Introduction to Tensors, Tensor Decompositions, and Tensor PCA , 2018, Proceedings of the IEEE.

[7]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Hanghang Tong,et al.  Factor Matrix Trace Norm Minimization for Low-Rank Tensor Completion , 2014, SDM.

[9]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[10]  Alexander Shapiro,et al.  Matrix Completion With Deterministic Pattern: A Geometric Perspective , 2018, IEEE Transactions on Signal Processing.

[11]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[12]  Robert D. Nowak,et al.  A converse to low-rank matrix completion , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[13]  Yi Shen,et al.  Structure tensor total variation-regularized weighted nuclear norm minimization for hyperspectral image mixed denoising , 2017, Signal Process..

[14]  Prateek Jain,et al.  Universal Matrix Completion , 2014, ICML.

[15]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[16]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[17]  Peter Lancaster,et al.  Norms on direct sums and tensor products , 1972 .

[18]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[19]  Gideon Schechtman,et al.  Deterministic algorithms for matrix completion , 2014, Random Struct. Algorithms.

[20]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[21]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[22]  R. Bro,et al.  PARAFAC and missing values , 2005 .

[23]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[24]  T. Tony Cai,et al.  Matrix completion via max-norm constrained optimization , 2013, ArXiv.

[25]  Dehui Yang,et al.  Weighted Matrix Completion and Recovery With Prior Subspace Information , 2018, IEEE Transactions on Information Theory.

[26]  David F. Gleich,et al.  Rank aggregation via nuclear norm minimization , 2011, KDD.

[27]  Prateek Jain,et al.  Provable Tensor Factorization with Missing Data , 2014, NIPS.

[28]  Xiaodong Wang,et al.  Rank Determination for Low-Rank Data Completion , 2017, J. Mach. Learn. Res..

[29]  J. Kruskal.  Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[30]  Xiaodong Wang,et al.  Fundamental Conditions for Low-CP-Rank Tensor Completion , 2017, J. Mach. Learn. Res..

[31]  James Caverlee,et al.  Tensor Completion Algorithms in Big Data Analytics , 2017, ACM Trans. Knowl. Discov. Data.

[32]  Nigel Boston,et al.  A characterization of deterministic sampling patterns for low-rank matrix completion , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[33]  Ankur Moitra,et al.  Noisy tensor completion via the sum-of-squares hierarchy , 2015, Mathematical Programming.

[34]  Yunming Ye,et al.  Low-Rank Tensor Completion with Total Variation for Visual Data Inpainting , 2017, AAAI.

[35]  Panagiotis Symeonidis,et al.  Tag recommendations based on tensor dimensionality reduction , 2008, RecSys '08.

[36]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[37]  Fast and Robust Spectrally Sparse Signal Recovery: A Provable Non-Convex Approach via Robust Low-Rank Hankel Matrix Reconstruction , 2019, ArXiv.

[38]  Troy Lee,et al.  Matrix Completion From any Given Set of Observations , 2013, NIPS.

[39]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[40]  R. Bro,et al.  PARAFAC2—Part I. A direct fitting algorithm for the PARAFAC2 model , 1999 .

[41]  Jian-Feng Cai,et al.  Accelerated Structured Alternating Projections for Robust Spectrally Sparse Signal Recovery , 2021, IEEE Transactions on Signal Processing.

[42]  David Suter,et al.  Recovering the missing components in a large noisy low-rank matrix: application to SFM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Gang Liu,et al.  Tensor completion using total variation and low-rank matrix factorization , 2016, Inf. Sci..

[44]  Junzhou Huang,et al.  Accelerated Dynamic MRI Reconstruction with Total Variation and Nuclear Norm Regularization , 2015, MICCAI.

[45]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[46]  Franz J. Király,et al.  The algebraic combinatorial approach for low-rank matrix completion , 2012, J. Mach. Learn. Res..

[47]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[48]  J. Leeuw,et al.  Principal component analysis of three-mode data by means of alternating least squares algorithms , 1980 .

[49]  Sourav Chatterjee.  A Deterministic Theory of Low Rank Matrix Completion , 2020, IEEE Transactions on Information Theory.

[50]  Juan Carlos Niebles,et al.  Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification , 2010, ECCV.

[51]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[52]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[53]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[54]  Nan Zhang,et al.  Uncovering the Spatio-Temporal Dynamics of Memes in the Presence of Incomplete Information , 2016, CIKM.

[55]  R. Bro.  PARAFAC. Tutorial and applications , 1997 .

[56]  Sudhish N. George,et al.  Twist tensor total variation regularized-reweighted nuclear norm based tensor completion for video missing area recovery , 2018, Inf. Sci..

[57]  Ali Taylan Cemgil,et al.  Link prediction in heterogeneous data via generalized coupled tensor factorization , 2013, Data Mining and Knowledge Discovery.

[58]  Deanna Needell,et al.  Weighted Matrix Completion From Non-Random, Non-Uniform Sampling Patterns , 2019, IEEE Transactions on Information Theory.

[59]  Yuanzhi Li,et al.  Recovery guarantee of weighted low-rank approximation via alternating minimization , 2016, ICML.

[60]  Bülent Yener,et al.  Unsupervised Multiway Data Analysis: A Literature Survey , 2009, IEEE Transactions on Knowledge and Data Engineering.

[61]  Xiaolan Liu,et al.  A Sequentially Truncated Higher Order Singular Value Decomposition-Based Algorithm for Tensor Completion , 2019, IEEE Transactions on Cybernetics.

[62]  Deanna Needell,et al.  Tensor Completion through Total Variation with Initialization from Weighted HOSVD , 2020, 2020 Information Theory and Applications Workshop (ITA).

[63]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[64]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[65]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.