A Bayesian approach for noisy matrix completion: Optimal rate under general sampling distribution

Bayesian methods for low-rank matrix completion with noise have been shown to be very efficient computationally. While the behaviour of penalized minimization methods is well understood both from the theoretical and computational points of view in this problem, the theoretical optimality of Bayesian estimators have not been explored yet. In this paper, we propose a Bayesian estimator for matrix completion under general sampling distribution. We also provide an oracle inequality for this estimator. This inequality proves that, whatever the rank of the matrix to be estimated, our estimator reaches the minimax-optimal rate of convergence (up to a logarithmic factor). We end the paper with a short simulation study.

[1]  John Shawe-Taylor,et al.  A PAC analysis of a Bayesian estimator , 1997, COLT '97.

[2]  Petar M. Djuric,et al.  Gibbs sampling approach for generation of truncated multivariate Gaussian random variables , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[3]  Olivier Catoni,et al.  Statistical learning theory and stochastic optimization , 2004 .

[4]  David A. McAllester.  Some PAC-Bayesian Theorems , 1998, COLT' 98.

[5]  O. Catoni.  A PAC-Bayesian approach to adaptive classification , 2004 .

[6]  James Bennett,et al.  The Netflix Prize , 2007 .

[7]  O. Catoni.  PAC-BAYESIAN SUPERVISED CLASSIFICATION: The Thermodynamics of Statistical Learning , 2007, 0712.0248.

[8]  Yew Jin Lim.  Variational Bayesian Approach to Movie Rating Prediction , 2007 .

[9]  Ruslan Salakhutdinov,et al.  Bayesian probabilistic matrix factorization using Markov chain Monte Carlo , 2008, ICML '08.

[10]  Arnak S. Dalalyan,et al.  Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity , 2008, Machine Learning.

[11]  Neil D. Lawrence,et al.  Non-linear matrix factorization with Gaussian processes , 2009, ICML '09.

[12]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[13]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[14]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[15]  L. Carin,et al.  Nonparametric Bayesian matrix completion , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[16]  Ohad Shamir,et al.  Learning with the weighted trace-norm under arbitrary sampling distributions , 2011, NIPS.

[17]  Karim Lounici,et al.  Pac-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights , 2010, 1009.2707.

[18]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[19]  Martin J. Wainwright,et al.  Restricted strong convexity and weighted matrix completion: Optimal bounds with noise , 2010, J. Mach. Learn. Res..

[20]  Pierre Alquier,et al.  Bayesian Methods for Low-Rank Matrix Estimation: Short Survey and Theoretical Study , 2013, ALT.

[21]  Pierre Alquier,et al.  Sparse single-index model , 2011, J. Mach. Learn. Res..

[22]  Christopher Ré,et al.  Parallel stochastic gradient algorithms for large-scale matrix completion , 2013, Mathematical Programming Computation.

[23]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[24]  Taiji Suzuki.  Convergence rate of Bayesian tensor estimator: Optimal rate without restricted strong convexity , 2014, ArXiv.

[25]  N. Chopin,et al.  Bayesian matrix completion: prior specification , 2014, 1406.1440.

[26]  O. Klopp.  Noisy low-rank matrix completion with general sampling distribution , 2012, 1203.0108.

[27]  Judith Rousseau,et al.  Bayesian matrix completion: prior specification and consistency , 2014 .