De-biasing low-rank projection for matrix completion

We study matrix completion with non-uniform, deterministic sampling patterns. We introduce a computable parameter, which is a function of the sampling pattern, and show that if this parameter is small, then we may recover missing entries of the matrix, with appropriate weights. We theoretically analyze a simple and well-known recovery method, which simply projects the (zero-padded) subsampled matrix onto the set of low-rank matrices. We show that under non-uniform deterministic sampling, this method yields a biased solution, and we propose an algorithm to de-bias it. Numerical simulations demonstrate that de-biasing significantly improves the estimate. However, when the observations are noisy, the error of this method can be sub-optimal when the sampling is highly non-uniform. To remedy this, we suggest an alternative which is based on projection onto the max-norm ball whose robustness to noise tolerates arbitrarily non-uniform sampling. Finally, we analyze convex optimization in this framework.

[1]  Alexandre d'Aspremont,et al.  Optimal Solutions for Sparse Principal Component Analysis , 2007, J. Mach. Learn. Res..

[2]  O. Klopp Noisy low-rank matrix completion with general sampling distribution , 2012, 1203.0108.

[3]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[4]  David Suter,et al.  Recovering the missing components in a large noisy low-rank matrix: application to SFM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  O. Perron Zur Theorie der Matrices , 1907 .

[6]  Yoram Bresler,et al.  Computing performance guarantees for compressed sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Francis R. Bach,et al.  Low-rank matrix factorization with attributes , 2006, ArXiv.

[8]  Dehui Yang,et al.  Weighted Matrix Completion and Recovery With Prior Subspace Information , 2018, IEEE Transactions on Information Theory.

[9]  Emmanuel J. Candès,et al.  Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements , 2011, IEEE Transactions on Information Theory.

[10]  Inderjit S. Dhillon,et al.  Matrix Completion from Power-Law Distributed Samples , 2009, NIPS.

[11]  Michael B. Wakin,et al.  MC^2: A Two-Phase Algorithm for Leveraged Matrix Completion , 2016, ArXiv.

[12]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[13]  S. Gaiffas,et al.  High Dimensional Matrix Estimation With Unknown Variance Of The Noise , 2011, 1112.3055.

[14]  Gitta Kutyniok,et al.  1 . 2 Sparsity : A Reasonable Assumption ? , 2012 .

[15]  Ewout van den Berg,et al.  1-Bit Matrix Completion , 2012, ArXiv.

[16]  A. Tsybakov,et al.  Estimation of high-dimensional low-rank matrices , 2009, 0912.5338.

[17]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[18]  V. Koltchinskii Von Neumann Entropy Penalization and Low Rank Matrix Estimation , 2010, 1009.2439.

[19]  Ruslan Salakhutdinov,et al.  Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm , 2010, NIPS.

[20]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[21]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[22]  O. Klopp Rank penalized estimators for high-dimensional matrices , 2011, 1104.1244.

[23]  Yinyu Ye,et al.  Semidefinite programming based algorithms for sensor network localization , 2006, TOSN.

[24]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[25]  M. Davenport ONE-BIT MATRIX COMPLETION FOR PAIRWISE COMPARISON MATRICES , 2013 .

[26]  Charles R. Johnson,et al.  The singular values of a Hadamard product : a basic inequality , 1987 .

[27]  Dustin G. Mixon,et al.  Certifying the Restricted Isometry Property is Hard , 2012, IEEE Transactions on Information Theory.

[28]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[29]  Alexandre d'Aspremont,et al.  Testing the nullspace property using semidefinite programming , 2008, Math. Program..

[30]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[31]  Yoav Seginer,et al.  The Expected Norm of Random Matrices , 2000, Combinatorics, Probability and Computing.

[32]  Amit Singer,et al.  Uniqueness of Low-Rank Matrix Completion by Rigidity Theory , 2009, SIAM J. Matrix Anal. Appl..

[33]  Troy Lee,et al.  Matrix Completion From any Given Set of Observations , 2013, NIPS.

[34]  慧 廣瀬 A Mathematical Introduction to Compressive Sensing , 2015 .

[35]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[36]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[37]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[38]  Thierry Bertin-Mahieux,et al.  The Million Song Dataset , 2011, ISMIR.

[39]  Yuanzhi Li,et al.  Recovery guarantee of weighted low-rank approximation via alternating minimization , 2016, ICML.

[40]  Cyrus Rashtchian Bounded Matrix Rigidity and John's Theorem , 2016, Electron. Colloquium Comput. Complex..

[41]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[42]  Arkadi Nemirovski,et al.  On verifiable sufficient conditions for sparse signal recovery via ℓ1 minimization , 2008, Math. Program..

[43]  David F. Gleich,et al.  Rank aggregation via nuclear norm minimization , 2011, KDD.

[44]  Gideon Schechtman,et al.  Deterministic algorithms for matrix completion , 2014, Random Struct. Algorithms.

[45]  Amit Singer,et al.  A remark on global positioning from local distances , 2008, Proceedings of the National Academy of Sciences.

[46]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[47]  Prateek Jain,et al.  Tighter Low-rank Approximation via Sampling the Leveraged Element , 2015, SODA.

[48]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[49]  G. Jameson Summing and nuclear norms in Banach space theory , 1987 .

[50]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[51]  Prateek Jain,et al.  Universal Matrix Completion , 2014, ICML.

[52]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[53]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..