Low-Rank Matrix Completion with Adversarial Missing Entries

We show that, so long as the number of missing entries in any row or column is bounded by a function of the dimension, rank, and incoherence of the matrix, nuclear norm minimization recovers the target matrix exactly. The range for which this guarantee holds is surprisingly large--in an $n\times n$ matrix of constant rank, there may be as many as $\Omega(n)$ entries missing in every row and column. Conversely, if only constantly many entries are missing in any row or column, then we may recover matrices of rank $\Omega(n)$. We also use adversarial matrix completion to give an algorithm for completing an order-$m$ symmetric low-rank tensor from its multilinear entries in time roughly proportional to the number of tensor entries. We apply our tensor completion algorithm to the problem of learning mixtures of product distributions over the hypercube, obtaining new algorithmic results. If the centers of the product distribution are linearly independent, then we recover distributions with as many as $\Omega(n)$ centers in polynomial time and sample complexity. In the general case, we recover distributions with as many as $\tilde\Omega(n)$ centers in quasi-polynomial time, answering an open problem of Feldman et al. (SIAM J. Comp.) for the special case of distributions with incoherent bias vectors.

[1]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[2]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[3]  Prateek Jain,et al.  Learning Mixtures of Discrete Product Distributions using Spectral Decompositions , 2013, COLT.

[4]  J. Feldman,et al.  Learning mixtures of product distributions over discrete domains , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[5]  Anton van den Hengel,et al.  Semidefinite Programming , 2014, Computer Vision, A Reference Guide.

[6]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[7]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[8]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[9]  Sanjeev Arora,et al.  Learning Topic Models -- Going beyond SVD , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[10]  Mary Wootters,et al.  Fast matrix completion without the condition number , 2014, COLT.

[11]  Pablo A. Parrilo,et al.  Diagonal and Low-Rank Matrix Decompositions, Correlation Matrices, and Ellipsoid Fitting , 2012, SIAM J. Matrix Anal. Appl..

[12]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[13]  Oded Regev,et al.  On lattices, learning with errors, random linear codes, and cryptography , 2005, STOC '05.

[14]  Satish Rao,et al.  Learning Mixtures of Product Distributions Using Correlations and Independence , 2008, COLT.

[15]  Anima Anandkumar,et al.  A tensor approach to learning mixed membership community models , 2013, J. Mach. Learn. Res..

[16]  Yishay Mansour,et al.  Estimating a mixture of two product distributions , 1999, COLT '99.

[17]  Anima Anandkumar,et al.  Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates , 2014, ArXiv.

[18]  Justin P. Haldar,et al.  Rank-Constrained Solutions to Linear Matrix Equations Using PowerFactorization , 2009, IEEE Signal Processing Letters.

[19]  G. Watson.  Characterization of the subdifferential of some matrix norms , 1992 .

[20]  Yehuda Koren,et al.  Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[21]  Anima Anandkumar,et al.  A Method of Moments for Mixture Models and Hidden Markov Models , 2012, COLT.

[22]  Ryan O'Donnell,et al.  Learning functions of k relevant variables , 2004, J. Comput. Syst. Sci..

[23]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[24]  René Peeters,et al.  Orthogonal representations over finite fields and the chromatic number of graphs , 1996, Comb..

[25]  Sham M. Kakade,et al.  Learning mixtures of spherical gaussians: moment methods and spectral decompositions , 2012, ITCS '13.

[26]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[27]  Furong Huang,et al.  Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition , 2015, COLT.

[28]  Vitaly Feldman.  Attribute-Efficient and Non-adaptive Learning of Parities and DNF Expressions , 2007, J. Mach. Learn. Res..

[29]  Joydeep Ghosh,et al.  Noisy Matrix Completion Using Alternating Minimization , 2013, ECML/PKDD.

[30]  Anima Anandkumar,et al.  Analyzing Tensor Power Method Dynamics: Applications to Learning Overcomplete Latent Variable Models , 2014, ArXiv.

[31]  Stephen J. Wright,et al.  Online algorithms for factorization-based structure from motion , 2013, IEEE Winter Conference on Applications of Computer Vision.

[32]  Moritz Hardt,et al.  Understanding Alternating Minimization for Matrix Completion , 2013, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[33]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[34]  Gregory Valiant.  Finding Correlations in Subquadratic Time, with Applications to Learning Parities and the Closest Pair Problem , 2015, J. ACM.