Robust Bayesian Matrix Factorisation

We analyse the noise arising in collaborative filtering when formalised as a probabilistic matrix factorisation problem. We show empirically that modelling row- and column-specific variances is important, the noise being in general non-Gaussian and heteroscedastic. We also advocate for the use of a Student-t prior for the latent features as the standard Gaussian is included as a special case. We derive several variational inference algorithms and estimate the hyperparameters by type-II maximum likelihood. Experiments on real data show that the predictive performance is significantly improved.

[1]  Michel Verleysen,et al.  Robust Bayesian clustering , 2007, Neural Networks.

[2]  B. Jørgensen.  Statistical Properties of the Generalized Inverse Gaussian Distribution , 1981 .

[3]  Thomas Hofmann,et al.  Latent Class Models for Collaborative Filtering , 1999, IJCAI.

[4]  Peter Filzmoser,et al.  Regressions , 2019, Energy Transfers by Convection.

[5]  Wei Chu,et al.  Gaussian Processes for Ordinal Regression , 2005, J. Mach. Learn. Res..

[6]  Neil D. Lawrence,et al.  Variational inference for Student-t models: Robust Bayesian interpolation and generalised component analysis , 2005, Neurocomputing.

[7]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[8]  Michel Verleysen,et al.  Robust probabilistic projections , 2006, ICML.

[9]  Juha Karhunen,et al.  Principal Component Analysis for Sparse High-Dimensional Data , 2007, ICONIP.

[10]  Thomas Hofmann,et al.  Robust collaborative filtering , 2007, RecSys '07.

[11]  James Bennett,et al.  The Netflix Prize , 2007 .

[12]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[13]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[14]  Michel Verleysen,et al.  Collaborative Filtering with interlaced Generalized Linear Models , 2008, ESANN.

[15]  Mohammad Emtiyaz Khan,et al.  Variational bounds for mixed-data factor analysis , 2010, NIPS.

[16]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[17]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[18]  Wim Wiegerinck,et al.  Variational Approximations between Mean Field Theory and the Junction Tree Algorithm , 2000, UAI.

[19]  Ruslan Salakhutdinov,et al.  Bayesian probabilistic matrix factorization using Markov chain Monte Carlo , 2008, ICML '08.

[20]  D. Rubin,et al.  ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME , 1999 .

[21]  Yee Whye Teh,et al.  Variational Bayesian Approach to Movie Rating Prediction , 2007, KDD 2007.

[22]  Victor J. Yohai,et al.  Robust Low-Rank Approximation of Data Matrices With Elementwise Contamination , 2008, Technometrics.

[23]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[24]  John Eccleston,et al.  Statistics and Computing , 2006 .

[25]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[26]  Shinichi Nakajima,et al.  Implicit Regularization in Variational Bayesian Matrix Factorization , 2010, ICML.

[27]  S. Zamir,et al.  Lower Rank Approximation of Matrices by Least Squares With Any Choice of Weights , 1979 .