LLORMA: Local Low-Rank Matrix Approximation

Matrix approximation is a common tool in recommendation systems, text mining, and computer vision. A prevalent assumption in constructing matrix approximations is that the partially observed matrix is low-rank. In this paper, we propose, analyze, and experiment with two procedures, one parallel and the other global, for constructing local matrix approximations. The two approaches approximate the observed matrix as a weighted sum of low-rank matrices. These matrices are limited to a local region of the observed matrix. We analyze the accuracy of the proposed local low-rank modeling. Our experiments show improvements in prediction accuracy over classical approaches for recommendation tasks.

[1]  Mingxuan Sun,et al.  PREA: personalized recommendation algorithms toolkit , 2012, J. Mach. Learn. Res..

[2]  Ohad Shamir,et al.  Large-Scale Convex Minimization with a Low-Rank Constraint , 2011, ICML.

[3]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[4]  Ruslan Salakhutdinov,et al.  Matrix reconstruction with the local max norm , 2012, NIPS.

[5]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[6]  Ruslan Salakhutdinov,et al.  Bayesian probabilistic matrix factorization using Markov chain Monte Carlo , 2008, ICML '08.

[7]  Nathan Srebro,et al.  Concentration-Based Guarantees for Low-Rank Matrix Reconstruction , 2011, COLT.

[8]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[9]  Ameet Talwalkar,et al.  Divide-and-Conquer Matrix Factorization , 2011, NIPS.

[10]  Michael J. Pazzani,et al.  Learning Collaborative Information Filters , 1998, ICML.

[11]  Neil D. Lawrence,et al.  Non-linear matrix factorization with Gaussian processes , 2009, ICML '09.

[12]  Mingxuan Sun,et al.  A Comparative Study of Collaborative Filtering Algorithms , 2012, Proceedings of the International Conference on Knowledge Discovery and Information Retrieval.

[13]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[14]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[15]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[16]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[17]  Gilad Lerman,et al.  Robust Locally Linear Analysis with Applications to Image Denoising and Blind Inpainting , 2013, SIAM J. Imaging Sci..

[18]  Yehuda Koren,et al.  Modeling relationships at multiple scales to improve accuracy of large recommender systems , 2007, KDD '07.

[19]  Mingxuan Sun,et al.  Automatic Feature Induction for Stagewise Collaborative Filtering , 2012, NIPS.

[20]  Dennis DeCoste,et al.  Collaborative prediction using ensembles of Maximum Margin Matrix Factorizations , 2006, ICML.

[21]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[22]  Ameet Talwalkar,et al.  Ensemble Nystrom Method , 2009, NIPS.

[23]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[24]  Charles X. Ling,et al.  Clustering-Based Matrix Factorization , 2013, ArXiv.

[25]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[26]  Joseph Sill,et al.  Feature-Weighted Linear Stacking , 2009, ArXiv.

[27]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[28]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.