Structural property-aware multilayer network embedding for latent factor analysis

Four novel multilayer network embedding algorithms are proposed with the consideration on four network structural properties.Four objective functions based on NMF are specially designed and optimized to preserve the target structural properties.Extensive experiments are conducted to evaluate the performances of the proposed embedding algorithms on different tasks. Multilayer network is a structure commonly used to describe and model the complex interaction between sets of entities/nodes. A three-layer example is the author-paper-word structure in which authors are linked by co-author relation, papers are linked by citation relation, and words are linked by semantic relation. Network embedding, which aims to project the nodes in the network into a relatively low-dimensional space for latent factor analysis, has recently emerged as an effective method for a variety of network-based tasks, such as collaborative filtering and link prediction. However, existing studies of network embedding both focus on the single-layer network and overlook the structural properties of the network, e.g., the degree distribution and communities, which are significant for node characterization, such as the preferences of users in a social network. In this paper, we propose four multilayer network embedding algorithms based on Nonnegative Matrix Factorization (NMF) with consideration given to four structural properties: whole network (NNMF), community (CNMF), degree distribution (DNMF), and max spanning tree (TNMF). Experiments on synthetic data show that the proposed algorithms are able to preserve the desired structural properties as designed. Experiments on real-world data show that multilayer network embedding improves the accuracy of document clustering and recommendation, and the four embedding algorithms corresponding to the four structural properties demonstrate the differences in performance on these two tasks. These results can be directly used in document clustering and recommendation systems.

[1]  David M. Blei,et al.  Relational Topic Models for Document Networks , 2009, AISTATS.

[2]  Krzysztof Janowicz,et al.  Linked Data, Big Data, and the 4th Paradigm , 2013, Semantic Web.

[3]  Xuelong Li,et al.  Constrained Nonnegative Matrix Factorization for Image Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jing Hua,et al.  Non-negative matrix factorization for semi-supervised data clustering , 2008, Knowledge and Information Systems.

[5]  Michael W Hadley,et al.  A new measure based on degree distribution that links information theory and network graph analysis , 2012, Neural systems & circuits.

[6]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[7]  Yihong Gong,et al.  Combining content and link for classification using matrix factorization , 2007, SIGIR.

[8]  Yu-Jin Zhang,et al.  Nonnegative Matrix Factorization: A Comprehensive Review , 2013, IEEE Transactions on Knowledge and Data Engineering.

[9]  Fei Wang,et al.  Community discovery using nonnegative matrix factorization , 2011, Data Mining and Knowledge Discovery.

[10]  Jie Lu,et al.  A trust-semantic fusion-based recommendation approach for e-business applications , 2012, Decis. Support Syst..

[11]  Lifeng Sun,et al.  Who should share what?: item-level social influence prediction for users and posts ranking , 2011, SIGIR.

[12]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[13]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Yousef Saad,et al.  Dense Subgraph Extraction with Application to Community Detection , 2012, IEEE Transactions on Knowledge and Data Engineering.

[15]  Shankar Subramaniam,et al.  A Newtonian Framework for Community Detection in Undirected Biological Networks , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[16]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[17]  Tim Berners-Lee,et al.  Linked data , 2020, Semantic Web for the Working Ontologist.

[18]  Jianping Fan,et al.  Multi-Layer Network for Influence Propagation over Microblog , 2012, PAISI.

[19]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[20]  Z. Wang,et al.  The structure and dynamics of multilayer networks , 2014, Physics Reports.

[21]  Ernesto Estrada,et al.  The Structure of Complex Networks: Theory and Applications , 2011 .

[22]  Tsvi Kuflik,et al.  Workshop on information heterogeneity and fusion in recommender systems (HetRec 2010) , 2010, RecSys '10.

[23]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[24]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[25]  Markus Flierl,et al.  Graph-Preserving Sparse Nonnegative Matrix Factorization With Application to Facial Expression Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Jane You,et al.  Low-rank matrix factorization with multiple Hypergraph regularizer , 2015, Pattern Recognit..

[27]  Lada A. Adamic,et al.  Power-Law Distribution of the World Wide Web , 2000, Science.

[28]  Jimeng Sun,et al.  Community Discovery via Metagraph Factorization , 2011, TKDD.

[29]  Yan Liu,et al.  Latent feature learning in social media network , 2013, ACM Multimedia.

[30]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[31]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[32]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[33]  Wei Cheng,et al.  Flexible and robust co-regularized multi-domain graph clustering , 2013, KDD.

[34]  Li Jia,et al.  Nonnegative Matrix Factorization With Regularizations , 2014, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[35]  Guangquan Zhang,et al.  Doubly Nonparametric Sparse Nonnegative Matrix Factorization Based on Dependent Indian Buffet Processes , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[37]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[38]  Tony Jebara,et al.  Structure preserving embedding , 2009, ICML '09.

[39]  Laurent Najman,et al.  Segmentation, Minimum Spanning Tree and Hierarchies , 2013 .

[40]  A. Arenas,et al.  Mathematical Formulation of Multilayer Networks , 2013, 1307.4977.

[41]  Damon Centola,et al.  The Spread of Behavior in an Online Social Network Experiment , 2010, Science.

[42]  Olivier L. de Weck,et al.  Multilayer network model for analysis and management of change propagation , 2011 .

[43]  William B. March,et al.  Fast euclidean minimum spanning tree: algorithm, analysis, and applications , 2010, KDD.

[44]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[45]  Mason A. Porter,et al.  Multilayer networks , 2013, J. Complex Networks.

[46]  Shilpa Chakravartula,et al.  Complex Networks: Structure and Dynamics , 2014 .

[47]  Xindong Wu,et al.  Nonnegative Matrix Factorization on Orthogonal Subspace , 2010, Pattern Recognit. Lett..

[48]  Charu C. Aggarwal,et al.  Heterogeneous Network Embedding via Deep Architectures , 2015, KDD.

[49]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[51]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[52]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[53]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.