A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method

Nonnegative matrix factorization (NMF)-based models possess fine representativeness of a target matrix, which is critically important in collaborative filtering (CF)-based recommender systems. However, current NMF-based CF recommenders suffer from the problem of high computational and storage complexity, as well as slow convergence rate, which prevents them from industrial usage in context of big data. To address these issues, this paper proposes an alternating direction method (ADM)-based nonnegative latent factor (ANLF) model. The main idea is to implement the ADM-based optimization with regard to each single feature, to obtain high convergence rate as well as low complexity. Both computational and storage costs of ANLF are linear with the size of given data in the target matrix, which ensures high efficiency when dealing with extremely sparse matrices usually seen in CF problems. As demonstrated by the experiments on large, real data sets, ANLF also ensures fast convergence and high prediction accuracy, as well as the maintenance of nonnegativity constraints. Moreover, it is simple and easy to implement for real applications of learning systems.

[1]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[2]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[3]  David Heckerman,et al.  Empirical Analysis of Predictive Algorithms for Collaborative Filtering , 1998, UAI.

[4]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[5]  Thomas G. Dietterich Ensemble Methods in Machine Learning , 2000, Multiple Classifier Systems.

[6]  John Riedl,et al.  Application of Dimensionality Reduction in Recommender System - A Case Study , 2000 .

[7]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[8]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[9]  Kenneth Y. Goldberg,et al.  Eigentaste: A Constant Time Collaborative Filtering Algorithm , 2001, Information Retrieval.

[10]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[11]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[12]  M. Wu,et al.  Collaborative Filtering via Ensembles of Matrix Factorizations , 2007, KDD 2007.

[13]  Gang Chen,et al.  Collaborative Filtering Using Orthogonal Nonnegative Matrix Tri-factorization , 2007, Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007).

[14]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[15]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[16]  Yifan Hu,et al.  Collaborative Filtering for Implicit Feedback Datasets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[17]  Wei Chu,et al.  Probabilistic Models for Incomplete Multi-dimensional Arrays , 2009, AISTATS.

[18]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[19]  Yihong Gong,et al.  Fast nonparametric matrix factorization for large-scale collaborative filtering , 2009, SIGIR.

[20]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[21]  Yehuda Koren,et al.  Collaborative filtering with temporal dynamics , 2009, KDD.

[22]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[23]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[24]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Shengcai Liao,et al.  Which photo groups should I choose? A comparative study of recommendation algorithms in Flickr , 2010, J. Inf. Sci..

[26]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[27]  MengChu Zhou,et al.  A Performance Modeling Scheme for Multistage Switch Networks With Phase-Type and Bursty Traffic , 2010, IEEE/ACM Transactions on Networking.

[28]  Ruslan Salakhutdinov,et al.  Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm , 2010, NIPS.

[29]  Michael R. Lyu,et al.  Learning to recommend with explicit and implicit social relations , 2011, TIST.

[30]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[31]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[32]  Giuseppe De Nicolao,et al.  Client–Server Multitask Learning From Distributed Datasets , 2008, IEEE Transactions on Neural Networks.

[33]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[34]  Guy Shani,et al.  Evaluating Recommendation Systems , 2011, Recommender Systems Handbook.

[35]  Zhang Xiong,et al.  Improving Latent Factor Model Based Collaborative Filtering via Integrated Folksonomy Factors , 2011, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[36]  Ning Zhou,et al.  A Hybrid Probabilistic Model for Unified Collaborative and Content-Based Image Tagging , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Josep Lluís de la Rosa i Esteva,et al.  A Negotiation-Style Recommender Based on Computational Ecology in Open Negotiation Environments , 2011, IEEE Transactions on Industrial Electronics.

[38]  Qiang Yang,et al.  Tracking Mobile Users in Wireless Networks via Semi-Supervised Colocalization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  MengChu Zhou,et al.  Understanding the evolution of a disaster—a Framework for Assessing Crisis in a System Environment (FACSE) , 2012, Natural Hazards.

[40]  Zoubin Ghahramani,et al.  Collaborative Gaussian Processes for Preference Learning , 2012, NIPS.

[41]  Djemel Ziou,et al.  Predictive Approach for User Long-Term Needs in Content-Based Image Suggestion , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[42]  MengChu Zhou,et al.  Impacts of 2.4-GHz ISM Band Interference on IEEE 802.15.4 Wireless Sensor Network Reliability in Buildings , 2012, IEEE Transactions on Instrumentation and Measurement.

[43]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Sophie Ahrens,et al.  Recommender Systems , 2012 .

[45]  Yung-Yu Chuang,et al.  Collaborative video reindexing via matrix factorization , 2012, TOMCCAP.

[46]  MengChu Zhou,et al.  Predicting Stay Time of Mobile Users With Contextual Information , 2013, IEEE Transactions on Automation Science and Engineering.

[47]  Zibin Zheng,et al.  Collaborative Web Service QoS Prediction via Neighborhood Integrated Matrix Factorization , 2013, IEEE Transactions on Services Computing.

[48]  Zibin Zheng,et al.  Predicting Quality of Service for Selection by Neighborhood-Based Collaborative Filtering , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[49]  Yixin Cao,et al.  Identifying overlapping communities as well as hubs and outliers via nonnegative matrix factorization , 2013, Scientific Reports.

[50]  Sotirios Chatzis,et al.  Nonparametric bayesian multitask collaborative filtering , 2013, CIKM.

[51]  MengChu Zhou,et al.  A Novel Method for Calculating Service Reputation , 2013, IEEE Transactions on Automation Science and Engineering.

[52]  Zoubin Ghahramani,et al.  Probabilistic Matrix Factorization with Non-random Missing Data , 2014, ICML.

[53]  Zhaohui Wu,et al.  An Efficient Recommendation Method for Improving Business Process Modeling , 2014, IEEE Transactions on Industrial Informatics.

[54]  MengChu Zhou,et al.  Optimal One-Wafer Cyclic Scheduling of Single-Arm Multicluster Tools With Two-Space Buffering Modules , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[55]  Xiaoguang Ma,et al.  Radio Channel Allocations With Global Optimality and Bounded Computational Scale , 2014, IEEE Transactions on Vehicular Technology.

[56]  Juan-Zi Li,et al.  Typicality-Based Collaborative Filtering Recommendation , 2014, IEEE Transactions on Knowledge and Data Engineering.

[57]  MengChu Zhou,et al.  An Indexing Network: Model and Applications , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[58]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[59]  MengChu Zhou,et al.  Service-Oriented Workflow Systems , 2015 .