An Inherently Nonnegative Latent Factor Model for High-Dimensional and Sparse Matrices from Industrial Applications

High-dimensional and sparse (HiDS) matrices are commonly encountered in many big-data-related and industrial applications like recommender systems. When acquiring useful patterns from them, nonnegative matrix factorization (NMF) models have proven to be highly effective owing to their fine representativeness of the nonnegative data. However, current NMF techniques suffer from: 1) inefficiency in addressing HiDS matrices; and 2) constraints in their training schemes. To address these issues, this paper proposes to extract nonnegative latent factors (NLFs) from HiDS matrices via a novel inherently NLF (INLF) model. It bridges the output factors and decision variables via a single-element-dependent mapping function, thereby making the parameter training unconstrained and compatible with general training schemes on the premise of maintaining the nonnegativity constraints. Experimental results on six HiDS matrices arising from industrial applications indicate that INLF is able to acquire NLFs from them more efficiently than any existing method does.

[1]  Qingsheng Zhu,et al.  Percentile Performance Estimation of Unreliable IaaS Clouds and Their Cost-Optimal Capacity Decision , 2017, IEEE Access.

[2]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[3]  Nicol N. Schraudolph,et al.  A Fast, Compact Approximation of the Exponential Function , 1999, Neural Computation.

[4]  Neil J. Hurley,et al.  Novelty and Diversity in Top-N Recommendation -- Analysis and Evaluation , 2011, TOIT.

[5]  MengChu Zhou,et al.  Stochastic Modeling and Performance Analysis of Migration-Enabled and Error-Prone Clouds , 2015, IEEE Transactions on Industrial Informatics.

[6]  Qingsheng Zhu,et al.  Incremental Collaborative Filtering recommender based on Regularized Matrix Factorization , 2012, Knowl. Based Syst..

[7]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[9]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[10]  Weixiong Zhang,et al.  Identification of hybrid node and link communities in complex networks , 2013, Scientific Reports.

[11]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[12]  Andrew M. Gross,et al.  Network-based stratification of tumor mutations , 2013, Nature Methods.

[13]  Yixin Cao,et al.  Identifying overlapping communities as well as hubs and outliers via nonnegative matrix factorization , 2013, Scientific Reports.

[14]  Shuai Li,et al.  Efficient Extraction of Non-negative Latent Factors from High-Dimensional and Sparse Matrices in Industrial Applications , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[15]  Lin Wu,et al.  A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Michael R. Lyu,et al.  Learning to recommend with social trust ensemble , 2009, SIGIR.

[17]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[18]  Yannick Deville,et al.  Linear-Quadratic Blind Source Separation Using NMF to Unmix Urban Hyperspectral Images , 2014, IEEE Transactions on Signal Processing.

[19]  Qiang Yang,et al.  Tracking Mobile Users in Wireless Networks via Semi-Supervised Colocalization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Changjun Jiang,et al.  Partition-based collaborative tensor factorization for POI recommendation , 2017, IEEE/CAA Journal of Automatica Sinica.

[21]  Mingsheng Shang,et al.  Symmetric Non-negative Latent Factor Models for Undirected Large Networks , 2017, IJCAI.

[22]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[23]  MengChu Zhou,et al.  An Efficient Second-Order Approach to Factorize Sparse Matrices in Recommender Systems , 2015, IEEE Transactions on Industrial Informatics.

[24]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[25]  Andrzej Cichocki,et al.  A Multiplicative Algorithm for Convolutive Non-Negative Matrix Factorization Based on Squared Euclidean Distance , 2009, IEEE Transactions on Signal Processing.

[26]  Yunni Xia,et al.  Applying the learning rate adaptation to the matrix factorization based collaborative filtering , 2013, Knowl. Based Syst..

[27]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[28]  Matei Zaharia,et al.  Matrix Computations and Optimization in Apache Spark , 2015, KDD.

[29]  Hareton K. N. Leung,et al.  A Highly Efficient Approach to Protein Interactome Mapping Based on Collaborative Filtering Framework , 2015, Scientific Reports.

[30]  Andy J. Wellings,et al.  Architecting Time-Critical Big-Data Systems , 2016, IEEE Transactions on Big Data.

[31]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[32]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[33]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[34]  Nathan Marz,et al.  Big Data: Principles and best practices of scalable realtime data systems , 2015 .

[35]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[36]  Zhu-Hong You,et al.  Using manifold embedding for assessing and predicting protein interactions from high-throughput experimental data , 2010, Bioinform..

[37]  P. Midgley,et al.  Three-dimensional imaging of localized surface plasmon resonances of metal nanoparticles , 2013, Nature.

[38]  H. Robbins,et al.  A Convergence Theorem for Non Negative Almost Supermartingales and Some Applications , 1985 .

[39]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[40]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[41]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[43]  MengChu Zhou,et al.  Incorporation of Efficient Second-Order Solvers Into Latent Factor Models for Accurate Prediction of Missing QoS Data , 2018, IEEE Transactions on Cybernetics.

[44]  MengChu Zhou,et al.  A Stochastic Approach to Analysis of Energy-Aware DVS-Enabled Cloud Datacenters , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  MengChu Zhou,et al.  An Incremental-and-Static-Combined Scheme for Matrix-Factorization-Based Collaborative Filtering , 2016, IEEE Transactions on Automation Science and Engineering.

[46]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[47]  Krzysztof C. Kiwiel,et al.  Convergence and efficiency of subgradient methods for quasiconvex minimization , 2001, Math. Program..

[48]  Qingsheng Zhu,et al.  A parallel matrix factorization based recommender by alternating stochastic gradient decent , 2012, Eng. Appl. Artif. Intell..

[49]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[50]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[51]  MengChu Zhou,et al.  Distributed Winner-Take-All in Dynamic Networks , 2017, IEEE Transactions on Automatic Control.

[52]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[53]  Di Wu,et al.  A Highly Accurate Framework for Self-Labeled Semisupervised Classification in Industrial Applications , 2018, IEEE Transactions on Industrial Informatics.

[54]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[55]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[56]  Diego Fernández,et al.  Comparison of collaborative filtering algorithms , 2011, ACM Trans. Web.

[57]  Stephen J. Wright,et al.  Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , 2011, NIPS.

[58]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[59]  Yuanqing Xia,et al.  Cloud control systems , 2015, IEEE/CAA Journal of Automatica Sinica.

[60]  MengChu Zhou,et al.  Stochastic Modeling and Quality Evaluation of Infrastructure-as-a-Service Clouds , 2015, IEEE Transactions on Automation Science and Engineering.

[61]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[62]  Shuai Li,et al.  Symmetric and Nonnegative Latent Factor Models for Undirected, High-Dimensional, and Sparse Networks in Industrial Applications , 2017, IEEE Transactions on Industrial Informatics.

[63]  Andy J. Wellings,et al.  Improving the predictability of distributed stream processors , 2015, Future Gener. Comput. Syst..

[64]  Zhaohui Wu,et al.  An Efficient Recommendation Method for Improving Business Process Modeling , 2014, IEEE Transactions on Industrial Informatics.

[65]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[66]  James Martens,et al.  Deep learning via Hessian-free optimization , 2010, ICML.

[67]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[68]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[69]  MengChu Zhou,et al.  Toward cloud computing QoS architecture: analysis of cloud systems and cloud services , 2017, IEEE/CAA Journal of Automatica Sinica.

[70]  MengChu Zhou,et al.  Dynamic Cloud Task Scheduling Based on a Two-Stage Strategy , 2018, IEEE Transactions on Automation Science and Engineering.

[71]  Paolo Avesani,et al.  Trust-aware recommender systems , 2007, RecSys '07.

[72]  MengChu Zhou,et al.  An Effective Scheme for QoS Estimation via Alternating Direction Method-Based Matrix Factorization , 2019, IEEE Transactions on Services Computing.