Elastic-net regularized latent factor analysis-based models for recommender systems

Abstract Latent factor analysis (LFA)-based models are highly efficient in recommender systems. The problem of LFA is defined on high-dimensional and sparse (HiDS) matrices corresponding to relationships among numerous entities in industrial applications. It is ill-posed without a unique and optimal solution, making regularization vital in improving the generality of an LFA-based model. Current models mostly adopt l2-norm-based regularization, which cannot regularize the latent factor distributions. For addressing this issue, this work applies the elastic-net-based regularization to an LFA-based model, thereby achieving an elastic-net regularized latent factor analysis-based (ERLFA) model. We further adopt two efficient learning algorithms, i.e., forward-looking sub-gradients and forward-backward splitting and stochastic proximal gradient descent, to train desired latent factors in an ERLFA-based model, resulting in two novel ERLFA-based models relying on different learning schemes. Experimental results on four large industrial datasets show that by regularizing the latent factor distribution, the proposed ERLFA-based models are able to achieve high prediction accuracy for missing data of an HiDS matrix without additional computational burden.

[1]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[2]  Chao Liu,et al.  Recommender systems with social regularization , 2011, WSDM '11.

[3]  Shuai Li,et al.  Efficient Extraction of Non-negative Latent Factors from High-Dimensional and Sparse Matrices in Industrial Applications , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[4]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[5]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[6]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[7]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[8]  Zidong Wang,et al.  Inferring nonlinear lateral flow immunoassay state-space models via an unscented Kalman filter , 2016, Science China Information Sciences.

[9]  Zidong Wang,et al.  Inference of Nonlinear State-Space Models for Sandwich-Type Lateral Flow Immunoassay Using Extended Kalman Filtering , 2011, IEEE Transactions on Biomedical Engineering.

[10]  Na Chen,et al.  Error Analysis for Matrix Elastic-Net Regularization Algorithms , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Ruiyun Yu,et al.  Minimizing Legal Exposure of High-Tech Companies through Collaborative Filtering Methods , 2016, KDD.

[12]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[13]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[14]  Zidong Wang,et al.  A Hybrid EKF and Switching PSO Algorithm for Joint State and Parameter Estimation of Lateral Flow Immunoassay Models , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[15]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[17]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Lina Yao,et al.  Unified Collaborative and Content-Based Web Service Recommendation , 2015, IEEE Transactions on Services Computing.

[19]  Hong Zhang,et al.  Denoising and deblurring gold immunochromatographic strip images via gradient projection algorithms , 2017, Neurocomputing.

[20]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[21]  Martha Larson,et al.  Collaborative Filtering beyond the User-Item Matrix , 2014, ACM Comput. Surv..

[22]  John Langford,et al.  Sparse Online Learning via Truncated Gradient , 2008, NIPS.

[23]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[24]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[25]  Zidong Wang,et al.  Image-Based Quantitative Analysis of Gold Immunochromatographic Strip via Cellular Neural Network Approach , 2014, IEEE Transactions on Medical Imaging.

[26]  Yoram Singer,et al.  Efficient Learning using Forward-Backward Splitting , 2009, NIPS.

[27]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[28]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[29]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[30]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[31]  MengChu Zhou,et al.  An Efficient Second-Order Approach to Factorize Sparse Matrices in Recommender Systems , 2015, IEEE Transactions on Industrial Informatics.

[32]  Yuan Qi,et al.  Nonparametric Bayesian Matrix Factorization by Power-EP , 2010, AISTATS.

[33]  Juan-Zi Li,et al.  Typicality-Based Collaborative Filtering Recommendation , 2014, IEEE Transactions on Knowledge and Data Engineering.

[34]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[35]  Qingsheng Zhu,et al.  Incremental Collaborative Filtering recommender based on Regularized Matrix Factorization , 2012, Knowl. Based Syst..

[36]  Thomas Hofmann,et al.  Latent semantic models for collaborative filtering , 2004, TOIS.

[37]  Huang Bai,et al.  A gradient-based algorithm for optimizing sensing matrix with normalization constraint , 2016, 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA).

[38]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[39]  MengChu Zhou,et al.  Incorporation of Efficient Second-Order Solvers Into Latent Factor Models for Accurate Prediction of Missing QoS Data , 2018, IEEE Transactions on Cybernetics.

[40]  Changjun Jiang,et al.  Partition-based collaborative tensor factorization for POI recommendation , 2017, IEEE/CAA Journal of Automatica Sinica.