Convergence Analysis of a Fast Non-negative Latent Factor Model

A fast non-negative latent factor (FNLF) model adopts a single latent factor-dependent, non-negative, multiplicative and momentum-incorporated update (SLF-NM2 U) algorithm, which can ensure fast convergence on a high-dimensional and sparse (HiDS) matrix according to empirical studies in prior researches. However, it is crucial to investigate the theoretical proof regarding the reason why incorporation of a generalized momentum method into an SLF-NM2 U algorithm can ensure the fast convergence of an FNLF model, which has not been addressed in previous work. Therefore, this paper aims to unveil how a generalized momentum method improves the convergence rate of an FNLF model in the discrete time case by combining physical analysis. The FNLF model is superior to the NLF model in terms of the convergence rate and the prediction accuracy of missing data. This conclusion is obtained by empirically research on the HiDS matrix in industrial applications, which also provides empirical basis for theoretical proof.

[1]  MengChu Zhou,et al.  Improved Quantum-Inspired Evolutionary Algorithm for Large-Size Lane Reservation , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[2]  Weixiong Zhang,et al.  Identification of hybrid node and link communities in complex networks , 2013, Scientific Reports.

[3]  MengChu Zhou,et al.  An Inherently Nonnegative Latent Factor Model for High-Dimensional and Sparse Matrices from Industrial Applications , 2018, IEEE Transactions on Industrial Informatics.

[4]  MengChu Zhou,et al.  An Improved Exact $\varepsilon$-Constraint and Cut-and-Solve Combined Method for Biobjective Robust Lane Reservation , 2015, IEEE Transactions on Intelligent Transportation Systems.

[5]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[6]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[7]  Andrew M. Gross,et al.  Network-based stratification of tumor mutations , 2013, Nature Methods.

[8]  Yixin Cao,et al.  Identifying overlapping communities as well as hubs and outliers via nonnegative matrix factorization , 2013, Scientific Reports.

[9]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[10]  MengChu Zhou,et al.  Non-Negativity Constrained Missing Data Estimation for High-Dimensional and Sparse Matrices from Industrial Applications , 2020, IEEE Transactions on Cybernetics.

[11]  Davide Heller,et al.  STRING v10: protein–protein interaction networks, integrated over the tree of life , 2014, Nucleic Acids Res..

[12]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[13]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[14]  MengChu Zhou,et al.  Incorporation of Efficient Second-Order Solvers Into Latent Factor Models for Accurate Prediction of Missing QoS Data , 2018, IEEE Transactions on Cybernetics.

[15]  MengChu Zhou,et al.  Temporal Pattern-Aware QoS Prediction via Biased Non-Negative Latent Factorization of Tensors , 2020, IEEE Transactions on Cybernetics.

[16]  Yung-Yu Chuang,et al.  Collaborative video reindexing via matrix factorization , 2012, TOMCCAP.

[17]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[18]  MengChu Zhou,et al.  Algorithms of Unconstrained Non-Negative Latent Factor Analysis for Recommender Systems , 2019, IEEE Transactions on Big Data.

[19]  MengChu Zhou,et al.  An Effective Scheme for QoS Estimation via Alternating Direction Method-Based Matrix Factorization , 2019, IEEE Transactions on Services Computing.

[20]  Mingsheng Shang,et al.  A Fast Non-Negative Latent Factor Model Based on Generalized Momentum Method , 2021, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[21]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[22]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Guisong Yang,et al.  Improved Symmetric and Nonnegative Matrix Factorization Models for Undirected, Sparse and Large-Scaled Networks: A Triple Factorization-Based Approach , 2020, IEEE Transactions on Industrial Informatics.

[24]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[25]  Zhu-Hong You,et al.  Using manifold embedding for assessing and predicting protein interactions from high-throughput experimental data , 2010, Bioinform..

[26]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[27]  Bir Bhanu,et al.  Semantic Concept Co-Occurrence Patterns for Image Annotation and Retrieval , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  MengChu Zhou,et al.  Centralized Charging Strategy and Scheduling Algorithm for Electric Vehicles Under a Battery Swapping Scenario , 2016, IEEE Transactions on Intelligent Transportation Systems.

[29]  Xiaochun Cao,et al.  A Unified Semi-Supervised Community Detection Framework Using Latent Space Graph Regularization , 2015, IEEE Transactions on Cybernetics.

[30]  MengChu Zhou,et al.  Latent Factor-Based Recommenders Relying on Extended Stochastic Gradient Descent Algorithms , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Qiang Yang,et al.  Tracking Mobile Users in Wireless Networks via Semi-Supervised Colocalization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.