Exact Matrix Completion via Convex Optimization

Suppose that one observes an incomplete subset of entries selected from a low-rank matrix. When is it possible to complete the matrix and recover the entries that have not been seen? We demonstrate that in very general settings, one can perfectly recover all of the missing entries from most sufficiently large subsets by solving a convex programming problem that finds the matrix with the minimum nuclear norm agreeing with the observed entries. The techniques used in this analysis draw upon parallels in the field of compressed sensing, demonstrating that objects other than signals and images can be perfectly reconstructed from very limited information.

[1]  Dima Grigoriev,et al.  Complexity of Quantifier Elimination in the Theory of Algebraically Closed Fields , 1984, MFCS.

[2]  Béla Bollobás,et al.  Random Graphs , 1985 .

[3]  Charles R. Johnson,et al.  The singular values of a Hadamard product : a basic inequality , 1987 .

[4]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[5]  G. Watson.  Characterization of the subdifferential of some matrix norms , 1992 .

[6]  V. Peña.  Decoupling and Khintchine's Inequalities for $U$-Statistics , 1992 .

[7]  V. Peña,et al.  Decoupling Inequalities for the Tail Probabilities of Multivariate $U$-Statistics , 1993, math/9309211.

[8]  N. Linial,et al.  The geometry of graphs and some of its algorithmic applications , 1994, FOCS.

[9]  M. Talagrand.  New concentration inequalities in product spaces , 1996 .

[10]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[11]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[12]  G. Papavassilopoulos,et al.  On the rank minimization problem over a positive semidefinite linear matrix inequality , 1997, IEEE Trans. Autom. Control..

[13]  C. Beck,et al.  Computational study and comparisons of LFT reducibility methods , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[14]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[15]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[16]  A. Buchholz.  Operator Khintchine inequality in non-commutative probability , 2001 .

[17]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[18]  Anna R. Karlin,et al.  Spectral analysis of data , 2001, STOC '01.

[19]  Kim-Chuan Toh,et al.  SDPT3 — a Matlab software package for semidefinite-quadratic-linear programming, version 3.0 , 2001 .

[20]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[21]  Adrian Lewis,et al.  The mathematics of eigenvalue optimization , 2003, Math. Program..

[22]  Alan M. Frieze,et al.  Fast monte-carlo algorithms for finding low-rank approximations , 2004, JACM.

[23]  Nathan Srebro,et al.  Learning with matrix factorizations , 2004 .

[24]  Anthony Man-Cho So,et al.  Theory of semidefinite programming for Sensor Network Localization , 2005, SODA '05.

[25]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[26]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[27]  E. Rio,et al.  Concentration around the mean for maxima of empirical processes , 2005, math/0506594.

[28]  Francis R. Bach,et al.  Low-rank matrix factorization with attributes , 2006, ArXiv.

[29]  S. Muthukrishnan,et al.  Subspace Sampling and Relative-Error Matrix Approximation: Column-Row-Based Methods , 2006, ESA.

[30]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[31]  S. Muthukrishnan,et al.  Subspace Sampling and Relative-Error Matrix Approximation: Column-Based Methods , 2006, APPROX-RANDOM.

[32]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[33]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[34]  Mark Rudelson,et al.  Sampling from large matrices: An approach through geometric functional analysis , 2005, JACM.

[35]  Per-Gunnar Martinsson,et al.  Randomized algorithms for the low-rank approximation of matrices , 2007, Proceedings of the National Academy of Sciences.

[36]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[37]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[38]  Jieping Ye,et al.  An accelerated gradient method for trace norm minimization , 2009, ICML '09.

[39]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[40]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[41]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[42]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[43]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[44]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[45]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[46]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[47]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[48]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[49]  Pablo A. Parrilo,et al.  Latent variable graphical model selection via convex optimization , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[50]  A. Willsky,et al.  The Convex algebraic geometry of linear inverse problems , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[51]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[52]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[53]  A. Willsky,et al.  Latent variable graphical model selection via convex optimization , 2010 .

[54]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[55]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[56]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[57]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[58]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[59]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[60]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[61]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[62]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.