Block based Singular Value Decomposition approach to matrix factorization for recommender systems

With the abundance of data in recent years, interesting challenges are posed in the area of recommender systems. Producing high quality recommendations with scalability and performance is the need of the hour. Singular Value Decomposition(SVD) based recommendation algorithms have been leveraged to produce better results. In this paper, we extend the SVD technique further for scalability and performance in the context of 1) multi-threading 2) multiple computational units (with the use of Graphical Processing Units) and 3) distributed computation. We propose block based matrix factorization (BMF) paired with SVD. This enabled us to take advantage of SVD over basic matrix factorization(MF) while taking advantage of parallelism and scalability through BMF. We used Compute Unified Device Architecture (CUDA) platform and related hardware for leveraging Graphical Processing Unit (GPU) along with block based SVD to demonstrate the advantages in terms of performance and memory.

[1]  Arun K. Pujari,et al.  Proximal maximum margin matrix factorization for collaborative filtering , 2017, Pattern Recognit. Lett..

[2]  Jengnan Tzeng,et al.  Split-and-Combine Singular Value Decomposition for Large-Scale Matrix , 2013, J. Appl. Math..

[3]  Yehuda Koren,et al.  The BellKor Solution to the Netflix Grand Prize , 2009 .

[4]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[5]  Inderjit S. Dhillon,et al.  Scalable Coordinate Descent Approaches to Parallel Matrix Factorization for Recommender Systems , 2012, 2012 IEEE 12th International Conference on Data Mining.

[6]  Xiaoqian Jiang,et al.  Fast and Robust Parallel SGD Matrix Factorization , 2015, KDD.

[7]  Ming-Hsuan Yang,et al.  Incremental Learning for Robust Visual Tracking , 2008, International Journal of Computer Vision.

[8]  Haesun Park,et al.  DC-NMF: nonnegative matrix factorization based on divide-and-conquer for fast clustering and topic modeling , 2017, J. Glob. Optim..

[9]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[10]  Tat-Jun Chin,et al.  Incremental kernel SVD for face recognition with image sets , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[11]  Binbin Pan,et al.  Block kernel Nonnegative Matrix Factorization and its application to Face Recognition , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[12]  Geoffrey J. McLachlan,et al.  A Very Fast Algorithm for Matrix Factorization , 2010, ArXiv.

[13]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[14]  Shi Feng,et al.  Localized matrix factorization for recommendation based on matrix block diagonal forms , 2013, WWW.

[15]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[16]  Chih-Jen Lin,et al.  A Learning-Rate Schedule for Stochastic Gradient Methods to Matrix Factorization , 2015, PAKDD.

[17]  Venu Satuluri,et al.  Factorbird - a Parameter Server Approach to Distributed Matrix Factorization , 2014, ArXiv.

[18]  Liana L. Fong,et al.  Matrix Factorization on GPUs with Memory Optimization and Approximate Computing , 2018, ICPP.

[19]  Yannis Sismanis,et al.  Sparkler: supporting large-scale matrix factorization , 2013, EDBT '13.

[20]  Karl Rupp,et al.  GPU-Accelerated Non-negative Matrix Factorization for Text Mining , 2012, NLDB.

[21]  G. Karypis,et al.  Incremental Singular Value Decomposition Algorithms for Highly Scalable Recommender Systems , 2002 .

[22]  L. Lathauwer,et al.  Dimensionality reduction in higher-order signal processing and rank-(R1,R2,…,RN) reduction in multilinear algebra , 2004 .

[23]  Inderjit S. Dhillon,et al.  NOMAD: Nonlocking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion , 2013, Proc. VLDB Endow..

[24]  Chih-Jen Lin,et al.  A fast parallel SGD for matrix factorization in shared memory systems , 2013, RecSys.

[25]  Sven Koitka,et al.  nmfgpu4R: GPU-Accelerated Computation of the Non-Negative Matrix Factorization (NMF) Using CUDA Capable Hardware , 2016, R J..

[26]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[27]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[28]  Arun K. Pujari,et al.  Collaborative filtering using multiple binary maximum margin matrix factorizations , 2017, Inf. Sci..