Non-negativity constrained missing data estimation for high-dimensional and sparse matrices

Latent factor (LF) models have proven to be accurate and efficient in extracting hidden knowledge from high-dimensional and sparse (HiDS) matrices. However, most LF models fail to fulfill the non-negativity constraints that reflect the non-negative nature of industrial data. Yet existing non-negative LF models for HiDS matrices suffer from slow convergence leading to considerable time cost. An alternating direction method-based non-negative latent factor (ANLF) model decomposes a non-negative optimization process into small sub-tasks. It updates each LF non-negatively based on the latest state of those trained before, thereby achieving fast convergence and maintaining high prediction accuracy and scalability. This paper theoretically analyze the characteristics of an ANLF model, and presents detailed empirical study regarding its performance on several HiDS matrices arising from industrial applications currently in use. Therefore, its capability of addressing HiDS matrices is validated in both theory and practice.

[1]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[3]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[4]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[5]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[6]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[7]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[9]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[10]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[11]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[12]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[13]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[14]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[15]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[16]  C. V. Ramamoorthy,et al.  Knowledge and Data Engineering , 1989, IEEE Trans. Knowl. Data Eng..

[17]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[18]  Michael I. Jordan,et al.  Bayesian Nonnegative Matrix Factorization with Stochastic Variational Inference , 2014, Handbook of Mixed Membership Models and Their Applications.

[19]  Yihong Gong,et al.  Fast nonparametric matrix factorization for large-scale collaborative filtering , 2009, SIGIR.

[20]  Zibin Zheng,et al.  QoS-Aware Web Service Recommendation by Collaborative Filtering , 2011, IEEE Transactions on Services Computing.

[21]  Zibin Zheng,et al.  Exploring Latent Features for Memory-Based QoS Prediction in Cloud Computing , 2011, 2011 IEEE 30th International Symposium on Reliable Distributed Systems.