Efficient Extraction of Non-negative Latent Factors from High-Dimensional and Sparse Matrices in Industrial Applications

High-dimensional and sparse (HiDS) matrices are commonly encountered in many big data-related industrial applications like recommender systems. When acquiring useful patterns from them, non-negative matrix factorization (NMF) models have proven to be highly effective because of their fine representativeness of non-negative data. However, current NMF techniques suffer from a) inefficiency in addressing HiDS matrices, and b) constrained training schemes lack of flexibility, extensibility and adaptability. To address these issues, this work proposes to factorize industrial-size sparse matrices via a novel Inherently Non-negative Latent Factor (INLF) model. It connects the output factors and decision variables via a single-element-dependent sigmoid function, thereby innovatively removing the non-negativity constraints from its training process without impacting the solution accuracy. Hence, its training process is unconstrained, highly flexible and compatible with general learning schemes. Experimental results on five HiDS matrices generated by industrial applications indicate that INLF is able to acquire non-negative latent factors from them in a more efficient manner than any existing method does.

[1]  P. Midgley,et al.  Three-dimensional imaging of localized surface plasmon resonances of metal nanoparticles , 2013, Nature.

[2]  MengChu Zhou,et al.  An Incremental-and-Static-Combined Scheme for Matrix-Factorization-Based Collaborative Filtering , 2016, IEEE Transactions on Automation Science and Engineering.

[3]  MengChu Zhou,et al.  An Efficient Second-Order Approach to Factorize Sparse Matrices in Recommender Systems , 2015, IEEE Transactions on Industrial Informatics.

[4]  MengChu Zhou,et al.  Generating Highly Accurate Predictions for Missing QoS Data via Aggregating Nonnegative Latent Factor Models , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[6]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[7]  Weixiong Zhang,et al.  Identification of hybrid node and link communities in complex networks , 2013, Scientific Reports.

[8]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[9]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[10]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[11]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[12]  Zhu-Hong You,et al.  Using manifold embedding for assessing and predicting protein interactions from high-throughput experimental data , 2010, Bioinform..

[13]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[14]  Domonkos Tikk,et al.  Scalable Collaborative Filtering Approaches for Large Recommender Systems , 2009, J. Mach. Learn. Res..

[15]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[16]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[18]  MengChu Zhou,et al.  A Novel Approach to Extracting Non-Negative Latent Factors From Non-Negative Big Sparse Matrices , 2016, IEEE Access.

[19]  Lin Wu,et al.  A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[21]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[22]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[23]  Zhaohui Wu,et al.  An Efficient Recommendation Method for Improving Business Process Modeling , 2014, IEEE Transactions on Industrial Informatics.

[24]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[25]  Qiang Yang,et al.  Tracking Mobile Users in Wireless Networks via Semi-Supervised Colocalization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[27]  MengChu Zhou,et al.  A Nonnegative Latent Factor Model for Large-Scale Sparse Matrices in Recommender Systems via Alternating Direction Method , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Yannick Deville,et al.  Linear-Quadratic Blind Source Separation Using NMF to Unmix Urban Hyperspectral Images , 2014, IEEE Transactions on Signal Processing.

[29]  Vaclav Petricek,et al.  Recommender System for Online Dating Service , 2007, ArXiv.

[30]  Qingsheng Zhu,et al.  Incremental Collaborative Filtering recommender based on Regularized Matrix Factorization , 2012, Knowl. Based Syst..

[31]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Yin Zhang,et al.  An alternating direction algorithm for matrix completion with nonnegative factors , 2011, Frontiers of Mathematics in China.

[33]  Bradley N. Miller,et al.  GroupLens: applying collaborative filtering to Usenet news , 1997, CACM.

[34]  Diego Fernández,et al.  Comparison of collaborative filtering algorithms , 2011, ACM Trans. Web.

[35]  Ruslan Salakhutdinov,et al.  Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm , 2010, NIPS.

[36]  MengChu Zhou,et al.  An Efficient Non-Negative Matrix-Factorization-Based Approach to Collaborative Filtering for Recommender Systems , 2014, IEEE Transactions on Industrial Informatics.

[37]  Yixin Cao,et al.  Identifying overlapping communities as well as hubs and outliers via nonnegative matrix factorization , 2013, Scientific Reports.

[38]  Hyunsoo Kim,et al.  Sparse Non-negative Matrix Factorizations via Alternating Non-negativity-constrained Least Squares , 2006 .

[39]  Hareton K. N. Leung,et al.  A Highly Efficient Approach to Protein Interactome Mapping Based on Collaborative Filtering Framework , 2015, Scientific Reports.

[40]  Paolo Avesani,et al.  Trust-aware recommender systems , 2007, RecSys '07.

[41]  Andrzej Cichocki,et al.  A Multiplicative Algorithm for Convolutive Non-Negative Matrix Factorization Based on Squared Euclidean Distance , 2009, IEEE Transactions on Signal Processing.