Matrix Completion with Noisy Entries and Outliers

This paper considers the problem of matrix completion when the observed entries are noisy and contain outliers. It begins with introducing a new optimization criterion for which the recovered matrix is defined as its solution. This criterion uses the celebrated Huber function from the robust statistics literature to downweigh the effects of outliers. A practical algorithm is developed to solve the optimization involved. This algorithm is fast, straightforward to implement, and monotonic convergent. Furthermore, the proposed methodology is theoretically shown to be stable in a well defined sense. Its promising empirical performance is demonstrated via a sequence of simulation experiments, including image inpainting.

[1]  Constantine Caramanis,et al.  Robust Matrix Completion and Corrupted Columns , 2011, ICML.

[2]  Yiyuan She,et al.  Outlier Detection Using Nonconvex Penalized Regression , 2010, ArXiv.

[3]  James Bennett,et al.  The Netflix Prize , 2007 .

[4]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[5]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[6]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[7]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[8]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Trevor J. Hastie,et al.  Matrix completion and low-rank SVD via fast alternating least squares , 2014, J. Mach. Learn. Res..

[10]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[11]  A. Montanari,et al.  On positioning via distributed matrix completion , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[12]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[13]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[14]  D. Hunter,et al.  Optimization Transfer Using Surrogate Objective Functions , 2000 .

[15]  Goran Marjanovic,et al.  On $l_q$ Optimization and Matrix Completion , 2012, IEEE Transactions on Signal Processing.

[16]  Juha Karhunen,et al.  Bayesian Robust PCA of Incomplete Data , 2012, Neural Processing Letters.

[17]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[18]  Juha Karhunen,et al.  Robust PCA Methods for Complete and Missing Data , 2011 .

[19]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[20]  Douglas W. Nychka,et al.  The Role of Pseudo Data for Robust Smoothing with Application to Wavelet Regression , 2007 .

[21]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[22]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[23]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[24]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[25]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[26]  Kenneth Lange,et al.  Numerical analysis for statisticians , 1999 .

[27]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[28]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[29]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[30]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.