A NEW APPROACH OF GPU-ACCELERATED STOCHASTIC GRADIENT DESCENT METHOD FOR MATRIX FACTORIZATION

Matrix-factorization(MF-) based collaborative filtering (CF) is known to be an effective approach to recommendation, which has been widely used in many recommender systems. Stochastic gradient descent (SGD) is one of the most popular algorithms for solving MF-based CF. However, the large computational burden required by SGD poses a challenge of accelerating the SGD process. In the past few years, the graphics processing unit (GPU) has evolved into a very flexible and powerful computing resource. SGD methods for GPUs exist, and the main job is to find the parallelism in the calculation. However, existing parallel SGD approaches ignore the inherent parallelism of vector computation and so do not use the characteristic that a GPU is suitable for vector and matrix computing. In this paper, we aim to design an approach using the inherent parallelism of vector computation to exploit the large-scale parallelization features of a massively multi-threaded GPU to perform SGD. We make full use of the characteristic that GPU is suitable for vector and matrix computing to design the parallel SGD algorithm. Experimental results demonstrate that the proposed method can be well suited for the massively parallel GPU architecture and can outperform the existing method.

[1]  Xiaofeng Zhang,et al.  CPU versus GPU: which can perform matrix computation faster—performance comparison for basic linear algebra subprograms , 2018, Neural Computing and Applications.

[2]  Yun Liang,et al.  CuMF_SGD: Parallelized Stochastic Gradient Descent for Matrix Factorization on GPUs , 2017, HPDC.

[3]  Feng Liu,et al.  Joint Weighted Nonnegative Matrix Factorization for Mining Attributed Graphs , 2017, PAKDD.

[4]  P. Sadayappan,et al.  Parallel CCD++ on GPU for Matrix Factorization , 2017, GPGPU@PPoPP.

[5]  Jing Lin,et al.  GPUSGD: A GPU‐accelerated stochastic gradient descent algorithm for matrix factorization , 2016, Concurr. Comput. Pract. Exp..

[6]  Liana L. Fong,et al.  Faster and Cheaper: Parallelizing Large-Scale Matrix Factorization on GPUs , 2016, HPDC.

[7]  Yoram Singer,et al.  Train faster, generalize better: Stability of stochastic gradient descent , 2015, ICML.

[8]  Inderjit S. Dhillon,et al.  Parallel matrix factorization for recommender systems , 2014, Knowl. Inf. Syst..

[9]  Bin Wu,et al.  A Fast Distributed Stochastic Gradient Descent Algorithm for Matrix Factorization , 2014, BigMine.

[10]  Chih-Jen Lin,et al.  A fast parallel SGD for matrix factorization in shared memory systems , 2013, RecSys.

[11]  Rong Zheng,et al.  Asynchronous stochastic gradient descent for DNN training , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[13]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[14]  Diego Fernández,et al.  Comparison of collaborative filtering algorithms , 2011, ACM Trans. Web.

[15]  Domonkos Tikk,et al.  Fast als-based matrix factorization for explicit and implicit feedback datasets , 2010, RecSys '10.

[16]  Dennis M. Wilkinson,et al.  Large-Scale Parallel Collaborative Filtering for the Netflix Prize , 2008, AAIM.

[17]  Wen-mei W. Hwu,et al.  Optimization principles and application performance evaluation of a multithreaded GPU using CUDA , 2008, PPoPP.

[18]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[19]  V. P. Pauca,et al.  Nonnegative matrix factorization for spectral data analysis , 2006 .

[20]  Michael W. Berry,et al.  Document clustering using nonnegative matrix factorization , 2006, Inf. Process. Manag..

[21]  Patrice Y. Simard,et al.  Using GPUs for machine learning algorithms , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[22]  Tong Zhang,et al.  Solving large scale linear prediction problems using stochastic gradient descent algorithms , 2004, ICML.

[23]  Robin D. Burke,et al.  Hybrid Recommender Systems: Survey and Experiments , 2002, User Modeling and User-Adapted Interaction.

[24]  John Riedl,et al.  Recommender systems in e-commerce , 1999, EC '99.

[25]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[26]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.