Hyper-parameter-evolutionary latent factor analysis for high-dimensional and sparse data from recommender systems

Abstract High-dimensional and Sparse (HiDS) data generated by recommender systems (RSs) contain rich knowledge regarding users’ potential preferences. A Latent factor analysis (LFA) model enables efficient extraction of essential features from such data. However, an LFA model relies heavily on its hyper-parameters like learning rate and regularization coefficient, which must be chosen with care. However, traditional grid-search-based manual tuning is extremely time-consuming and computationally expensive. To address this issue, this study proposes a hyper-parameter-evolutionary latent factor analysis (HLFA) model. Its main idea is to build a swarm by taking the hyper-parameters of every single LFA-based model as particles, and then apply particle swarm optimization (PSO) to make its both hyper-parameters, i.e., the learning rate and regularization coefficient, self-adaptive according to a pre-defined fitness function. Experimental results on six HiDS matrices from real RSs indicate that an HLFA model outperforms several state-of-the-art LF models in terms of computational efficiency, and most importantly, without loss of prediction accuracy for missing data of an HiDS matrix.

[1]  Cong Wang,et al.  Dynamic Learning From Adaptive Neural Network Control of a Class of Nonaffine Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[3]  Jun Zhang,et al.  Genetic Learning Particle Swarm Optimization , 2016, IEEE Transactions on Cybernetics.

[4]  Fei Wang,et al.  Graph dual regularization non-negative matrix factorization for co-clustering , 2012, Pattern Recognit..

[5]  MengChu Zhou,et al.  Comprehensive Learning Particle Swarm Optimization Algorithm With Local Search for Multimodal Functions , 2019, IEEE Transactions on Evolutionary Computation.

[6]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[7]  Tom Schaul,et al.  Adaptive learning rates and parallelization for stochastic, sparse, non-smooth gradients , 2013, ICLR.

[8]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[9]  MengChu Zhou,et al.  Incorporation of Efficient Second-Order Solvers Into Latent Factor Models for Accurate Prediction of Missing QoS Data , 2018, IEEE Transactions on Cybernetics.

[10]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[11]  Nicola Barbieri,et al.  Probabilistic Approaches to Recommendations , 2014, Probabilistic Approaches to Recommendations.

[12]  Shuai Li,et al.  Symmetric and Nonnegative Latent Factor Models for Undirected, High-Dimensional, and Sparse Networks in Industrial Applications , 2017, IEEE Transactions on Industrial Informatics.

[13]  Paolo Avesani,et al.  Trust-Aware Collaborative Filtering for Recommender Systems , 2004, CoopIS/DOA/ODBASE.

[14]  Miquel Sànchez-Marrè,et al.  A Case-Based Recommendation Approach for Market Basket Data , 2015, IEEE Intelligent Systems.

[15]  Fuad E. Alsaadi,et al.  A Novel Switching Delayed PSO Algorithm for Estimating Unknown Parameters of Lateral Flow Immunoassay , 2016, Cognitive Computation.

[16]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[17]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Mingsheng Shang,et al.  A Fast Non-Negative Latent Factor Model Based on Generalized Momentum Method , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[19]  Kashi Nath Dey,et al.  Forecasting of software reliability using neighborhood fuzzy particle swarm optimization based novel neural network , 2019, IEEE/CAA Journal of Automatica Sinica.

[20]  Junjie Li,et al.  Rosenbrock artificial bee colony algorithm for accurate global optimization of numerical functions , 2011, Inf. Sci..

[21]  MengChu Zhou,et al.  An Efficient Second-Order Approach to Factorize Sparse Matrices in Recommender Systems , 2015, IEEE Transactions on Industrial Informatics.

[22]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[23]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[24]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[25]  Bo Li,et al.  Study on an improved adaptive PSO algorithm for solving multi-objective gate assignment , 2017, Applied Soft Computing.