Adaptive Regularization-Incorporated Latent Factor Analysis

The valuable knowledge contained in High-dimensional and Sparse (HiDS) matrices can be efficiently extracted by a latent factor (LF) model. Regularization techniques are widely incorporated into an LF model to avoid overfitting. The regularization coefficient is very crucial to the prediction accuracy of models. However, its tuning process is time-consuming and boring. This study aims at making the regularization coefficient of a regularized LF model self-adaptive. To do so, an adaptive particle swarm optimization (APSO) algorithm is introduced into a regularized LF model to automatically select the optimal regularization coefficient. Then, to enhance the global search capability of particles, we further propose an APSO and particle swarm optimization (PSO)-incorporated (AP) algorithm, thereby achieving an AP-based LF (APLF) model. Experimental results on four HiDS matrices generated by real applications demonstrate that an APLF model can achieve an automatic selection of regularization coefficient, and is superior to a regularized LF model in terms of prediction accuracy and computational efficiency.

[1]  Qing-Xian Wang,et al.  An adaptive latent factor model via particle swarm optimization , 2019, Neurocomputing.

[2]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[3]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[4]  Yihong Gong,et al.  Fast nonparametric matrix factorization for large-scale collaborative filtering , 2009, SIGIR.

[5]  Yixin Cao,et al.  Identifying overlapping communities as well as hubs and outliers via nonnegative matrix factorization , 2013, Scientific Reports.

[6]  Guoyin Wang,et al.  A Data-Aware Latent Factor Model for Web Service QoS Prediction , 2019, PAKDD.

[7]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Timos Sellis,et al.  Big data analytics in telecommunications: literature review and architecture recommendations , 2020, IEEE/CAA Journal of Automatica Sinica.

[9]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[10]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[11]  Yoram Singer,et al.  Efficient projections onto the l1-ball for learning in high dimensions , 2008, ICML '08.

[12]  MengChu Zhou,et al.  A Deep Latent Factor Model for High-Dimensional and Sparse Matrices in Recommender Systems , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Hareton K. N. Leung,et al.  A Highly Efficient Approach to Protein Interactome Mapping Based on Collaborative Filtering Framework , 2015, Scientific Reports.

[14]  Lexin Li,et al.  Regularized matrix regression , 2012, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[15]  Jun Zhang,et al.  Adaptive control of acceleration coefficients for particle swarm optimization based on clustering analysis , 2007, 2007 IEEE Congress on Evolutionary Computation.

[16]  MengChu Zhou,et al.  Temporal Pattern-Aware QoS Prediction via Biased Non-Negative Latent Factorization of Tensors , 2020, IEEE Transactions on Cybernetics.

[17]  Jia Chen,et al.  Randomized latent factor model for high-dimensional and sparse matrices from industrial applications , 2018, 2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC).

[18]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[19]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[20]  MengChu Zhou,et al.  An Inherently Nonnegative Latent Factor Model for High-Dimensional and Sparse Matrices from Industrial Applications , 2018, IEEE Transactions on Industrial Informatics.

[21]  Guoyin Wang,et al.  A Posterior-Neighborhood-Regularized Latent Factor Model for Highly Accurate Web Service QoS Prediction , 2022, IEEE Transactions on Services Computing.

[22]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[23]  Anup Basu,et al.  Graph regularized Lp smooth non-negative matrix factorization for data representation , 2019, IEEE/CAA Journal of Automatica Sinica.

[24]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[25]  Dimitri P. Bertsekas,et al.  Feature-based aggregation and deep reinforcement learning: a survey and some new implementations , 2018, IEEE/CAA Journal of Automatica Sinica.

[26]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[27]  Jiujun Cheng,et al.  Dendritic Neuron Model With Effective Learning Algorithms for Classification, Approximation, and Prediction , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[28]  MengChu Zhou,et al.  Incorporation of Efficient Second-Order Solvers Into Latent Factor Models for Accurate Prediction of Missing QoS Data , 2018, IEEE Transactions on Cybernetics.

[29]  Liwei Wang,et al.  Dropout Training, Data-dependent Regularization, and Generalization Bounds , 2018, ICML.

[30]  Michael R. Lyu,et al.  Learning to recommend with social trust ensemble , 2009, SIGIR.

[31]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[32]  Mohammad Ali Abbasi,et al.  Trust-Aware Recommender Systems , 2014 .

[33]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[34]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[35]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[36]  MengChu Zhou,et al.  Non-Negativity Constrained Missing Data Estimation for High-Dimensional and Sparse Matrices from Industrial Applications , 2020, IEEE Transactions on Cybernetics.

[37]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[38]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.