HONEM: Learning Embedding for Higher Order Networks

Representation learning on networks offers a powerful alternative to the oft painstaking process of manual feature engineering, and, as a result, has enjoyed considerable success in recent years. However, all the existing representation learning methods are based on the first-order network, that is, the network that only captures the pairwise interactions between the nodes. As a result, these methods may fail to incorporate non-Markovian higher order dependencies in the network. Thus, the embeddings that are generated may not accurately represent the underlying phenomena in a network, resulting in inferior performance in different inductive or transductive learning tasks. To address this challenge, this study presents higher order network embedding (HONEM), a higher order network (HON) embedding method that captures the non-Markovian higher order dependencies in a network. HONEM is specifically designed for the HON structure and outperforms other state-of-the-art methods in node classification, network reconstruction, link prediction, and visualization for networks that contain non-Markovian higher order dependencies.

[1]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[2]  Steven Skiena,et al.  Don't Walk, Skip!: Online Learning of Multi-scale Network Embeddings , 2016, ASONAM.

[3]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[4]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[5]  Ingo Scholtes,et al.  Causality-driven slow-down and speed-up of diffusion in non-Markovian temporal networks , 2013, Nature Communications.

[6]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[7]  Jingrui He,et al.  Motif-Preserving Dynamic Local Graph Cut , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[8]  Junjie Wu,et al.  Embedding Temporal Network via Neighborhood Formation , 2018, KDD.

[9]  Ingo Scholtes,et al.  Higher-order aggregate networks in the analysis of temporal networks: path structures and centralities , 2015, The European Physical Journal B.

[10]  Jian Pei,et al.  Arbitrary-Order Proximity Preserved Network Embedding , 2018, KDD.

[12]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[13]  Alexander J. Smola,et al.  Distributed large-scale natural graph factorization , 2013, WWW.

[14]  Mason A. Porter,et al.  Author Correction: The physics of spreading processes in multilayer networks , 2016, 1604.02021.

[15]  Jingrui He,et al.  A Local Algorithm for Structure-Preserving Graph Cut , 2017, KDD.

[16]  Ryan A. Rossi,et al.  Higher-order Network Representation Learning , 2018, WWW.

[17]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[18]  Jure Leskovec,et al.  Tensor Spectral Clustering for Partitioning Higher-order Network Structures , 2015, SDM.

[19]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[20]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[21]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[22]  Danai Koutra,et al.  On Proximity and Structural Role-based Embeddings in Networks , 2020, ACM Trans. Knowl. Discov. Data.

[23]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[24]  Martin Rosvall,et al.  Memory in network flows and its effects on spreading dynamics and community detection , 2013, Nature Communications.

[25]  Jian Li,et al.  Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec , 2017, WSDM.

[26]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[27]  Francesco Vaccarino,et al.  Topological Strata of Weighted Complex Networks , 2013, PloS one.

[28]  Hao Yu,et al.  MOHONE: Modeling Higher Order Network Effects in KnowledgeGraphs via Network Infused Embeddings , 2018, ArXiv.

[29]  Mason A. Porter,et al.  Multilayer networks , 2013, J. Complex Networks.

[30]  Jure Leskovec,et al.  Higher-order organization of complex networks , 2016, Science.

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[32]  Ryan A. Rossi,et al.  Dynamic Network Embeddings: From Random Walks to Temporal Random Walks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[33]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[34]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[35]  Nitesh V. Chawla,et al.  New perspectives and methods in link prediction , 2010, KDD.

[36]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[37]  Qiaozhu Mei,et al.  PTE: Predictive Text Embedding through Large-scale Heterogeneous Text Networks , 2015, KDD.

[38]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[39]  A. Arenas,et al.  Motif-based communities in complex networks , 2007, 0710.0059.

[40]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[41]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[42]  Nitesh V. Chawla,et al.  Representing higher-order dependencies in networks , 2015, Science Advances.

[43]  Jure Leskovec,et al.  Human wayfinding in information networks , 2012, WWW.

[44]  Steven Skiena,et al.  Walklets: Multiscale Graph Embeddings for Interpretable Network Classification , 2016, ArXiv.

[45]  Danai Koutra,et al.  From Community to Role-based Graph Embeddings , 2019, ArXiv.

[46]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[47]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[48]  Kevin Chen-Chuan Chang,et al.  Motif-based Convolutional Neural Network on Graphs , 2017, ArXiv.

[49]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[50]  Ingo Scholtes,et al.  When is a Network a Network?: Multi-Order Graphical Model Selection in Pathways and Temporal Networks , 2017, KDD.

[51]  Bruno Ribeiro,et al.  Detecting Anomalies in Sequential Data with Higher-order Networks , 2017, ArXiv.

[52]  Liang Gou,et al.  DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention Networks , 2020, WSDM.

[53]  Yueting Zhuang,et al.  Dynamic Network Embedding by Modeling Triadic Closure Process , 2018, AAAI.

[54]  Ingo Scholtes,et al.  Understanding Complex Systems: From Networks to Optimal Higher-Order Models , 2018, ArXiv.