PAC-Bayesian Matrix Completion with a Spectral Scaled Student Prior

We study the problem of matrix completion in this paper. A spectral scaled Student prior is exploited to favour the underlying low-rank structure of the data matrix. We provide a thorough theoretical investigation for our approach through PAC-Bayesian bounds. More precisely, our PAC-Bayesian approach enjoys a minimax-optimal oracle inequality which guarantees that our method works well under model misspecification and under general sampling distribution. Interestingly, we also provide efficient gradient-based sampling implementations for our approach by using Langevin Monte Carlo. More specifically, we show that our algorithms are significantly faster than Gibbs sampler in this problem. To illustrate the attractive features of our inference strategy, some numerical simulations are conducted and an application to image inpainting is demonstrated.

[1]  Pierre Alquier,et al.  On the properties of variational approximations of Gibbs posteriors , 2015, J. Mach. Learn. Res..

[2]  J. Geweke,et al.  Bayesian reduced rank regression in econometrics , 1996 .

[3]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[4]  Trevor J. Hastie,et al.  Matrix completion and low-rank SVD via fast alternating least squares , 2014, J. Mach. Learn. Res..

[5]  O. Klopp.  Noisy low-rank matrix completion with general sampling distribution , 2012, 1203.0108.

[6]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[7]  V. Koltchinskii,et al.  Nuclear norm penalization and optimal rates for noisy low rank matrix completion , 2010, 1011.6256.

[8]  Jukka Corander,et al.  Bayesian assessment of dimensionality in reduced rank regression , 2004 .

[9]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[10]  Ohad Shamir,et al.  Learning with the weighted trace-norm under arbitrary sampling distributions , 2011, NIPS.

[11]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[12]  Olivier Catoni,et al.  Statistical learning theory and stochastic optimization , 2004 .

[13]  Alexandre Bernardino,et al.  Matrix Completion for Weakly-Supervised Multi-Label Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Arnak S. Dalalyan,et al.  User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient , 2017, Stochastic Processes and their Applications.

[15]  Pierre Alquier,et al.  Bayesian Methods for Low-Rank Matrix Estimation: Short Survey and Theoretical Study , 2013, ALT.

[16]  A. Dalalyan.  Exponential weights in multivariate regression and a low-rankness favoring prior , 2018, Annales De L Institut Henri Poincare-probabilites Et Statistiques.

[17]  L. Carin,et al.  Nonparametric Bayesian matrix completion , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[18]  Karim Lounici,et al.  Pac-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights , 2010, 1009.2707.

[19]  Arnak S. Dalalyan,et al.  Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity , 2008, Machine Learning.

[20]  Aggelos K. Katsaggelos,et al.  Sparse Bayesian Methods for Low-Rank Matrix Estimation , 2011, IEEE Transactions on Signal Processing.

[21]  Pier Giovanni Bissiri,et al.  A general framework for updating belief distributions , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[22]  Arnak S. Dalalyan,et al.  Sparse Regression Learning by Aggregation and Langevin Monte-Carlo , 2009, COLT.

[23]  Thijs van Ommen,et al.  Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It , 2014, 1412.3730.

[24]  Pierre Alquier,et al.  1-Bit matrix completion: PAC-Bayesian analysis of a variational approximation , 2016, Machine Learning.

[25]  O. Catoni.  PAC-BAYESIAN SUPERVISED CLASSIFICATION: The Thermodynamics of Statistical Learning , 2007, 0712.0248.

[26]  Alain Durmus,et al.  High-dimensional Bayesian inference via the unadjusted Langevin algorithm , 2016, Bernoulli.

[27]  Benjamin Guedj,et al.  PAC-Bayesian high dimensional bipartite ranking , 2015, Journal of Statistical Planning and Inference.

[28]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[29]  Gary K. Chen,et al.  Genotype imputation via matrix completion , 2013, Genome research.

[30]  Yuling Yan,et al.  Inference and uncertainty quantification for noisy matrix completion , 2019, Proceedings of the National Academy of Sciences.

[31]  Yew Jin Lim.  Variational Bayesian Approach to Movie Rating Prediction , 2007 .

[32]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[33]  Richard Paap,et al.  Priors, posteriors and Bayes factors for a Bayesian analysis of cointegration , 1998 .

[34]  Judith Rousseau,et al.  Bayesian matrix completion: prior specification and consistency , 2014 .

[35]  Jian Sun,et al.  Image Completion Approaches Using the Statistics of Similar Patches , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  O. Klopp.  Matrix completion by singular value thresholding: sharp bounds , 2015, 1502.00146.

[37]  James Bennett,et al.  The Netflix Prize , 2007 .

[38]  Jun Fang,et al.  Fast Low-Rank Bayesian Matrix Completion With Hierarchical Gaussian Prior Models , 2017, IEEE Transactions on Signal Processing.

[39]  A. Bhattacharya,et al.  Bayesian fractional posteriors , 2016, The Annals of Statistics.

[40]  Xiuzhen Huang,et al.  SparRec: An effective matrix completion framework of missing data imputation for GWAS , 2016, Scientific reports.

[41]  Xi Chen,et al.  Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization , 2010, SDM.

[42]  Ruslan Salakhutdinov,et al.  Bayesian probabilistic matrix factorization using Markov chain Monte Carlo , 2008, ICML '08.

[43]  Gediminas Adomavicius,et al.  Context-aware recommender systems , 2008, RecSys '08.

[44]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[45]  Pierre Alquier,et al.  Concentration of tempered posteriors and of their variational approximations , 2017, The Annals of Statistics.