Online Dynamic Network Embedding

Network embedding is a very important method for network data. However, most of the algorithms can only deal with static networks. In this paper, we propose an algorithm Recurrent Neural Network Embedding (RNNE) to deal with dynamic network, which can be typically divided into two categories: a) topologically evolving graphs whose nodes and edges will increase (decrease) over time; b) temporal graphs whose edges contain time information. In order to handle the changing size of dynamic networks, RNNE adds virtual node, which is not connected to any other nodes, to the networks and replaces it when new node arrives, so that the network size can be unified at different time. On the one hand, RNNE pays attention to the direct links between nodes and the similarity between the neighborhood structures of two nodes, trying to preserve the local and global network structure. On the other hand, RNNE reduces the influence of noise by transferring the previous embedding information. Therefore, RNNE can take into account both static and dynamic characteristics of the network.We evaluate RNNE on five networks and compare with several state-of-the-art algorithms. The results demonstrate that RNNE has advantages over other algorithms in reconstruction, classification and link predictions.

[1]  X. Yao,et al.  Model-based kernel for efficient time series analysis , 2013, KDD.

[2]  Huanhuan Chen,et al.  Predictive Ensemble Pruning by Expectation Propagation , 2009, IEEE Transactions on Knowledge and Data Engineering.

[3]  Qiongkai Xu,et al.  GraRep: Learning Graph Representations with Global Structural Information , 2015, CIKM.

[4]  Nicolas Le Roux,et al.  Efficient Non-Parametric Function Induction in Semi-Supervised Learning , 2004, AISTATS.

[5]  M. Brand,et al.  Fast low-rank modifications of the thin singular value decomposition , 2006 .

[6]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[7]  Huanhuan Chen,et al.  Efficient Probabilistic Classification Vector Machine With Incremental Basis Function Selection , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Huanhuan Chen,et al.  Cognitive fault diagnosis in Tennessee Eastman Process using learning in the model space , 2014, Comput. Chem. Eng..

[9]  Huanhuan Chen,et al.  Scalable Graph-Based Semi-Supervised Learning through Sparse Bayesian Model , 2017, IEEE Transactions on Knowledge and Data Engineering.

[10]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[11]  Geoffrey E. Hinton,et al.  Semantic hashing , 2009, Int. J. Approx. Reason..

[12]  W. Stefansky.  Rejecting Outliers in Factorial Designs , 1972 .

[13]  Huanhuan Chen,et al.  Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[14]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[15]  Huanhuan Chen,et al.  Learning in the Model Space for Cognitive Fault Diagnosis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  F. E. Grubbs.  Procedures for Detecting Outlying Observations in Samples , 1969 .

[17]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[18]  Huanhuan Chen,et al.  Sequential data classification by dynamic state warping , 2017, Knowledge and Information Systems.

[19]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[20]  Zachary Chase Lipton.  A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[21]  Michiel E. Hochstenbach,et al.  A Jacobi-Davidson type method for the generalized singular value problem , 2009 .

[22]  Hanghang Tong,et al.  Fast Eigen-Functions Tracking on Dynamic Graphs , 2015, SDM.

[23]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[24]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[25]  Huanhuan Chen,et al.  Model Metric Co-Learning for Time Series Classification , 2015, IJCAI.

[26]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[27]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[28]  Huanhuan Chen,et al.  Symbolic Sequence Classification in the Fractal Space , 2018 .

[29]  Wenwu Zhu,et al.  DepthLGP: Learning Embeddings of Out-of-Sample Nodes in Dynamic Networks , 2018, AAAI.

[30]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[31]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.

[32]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[33]  Huanhuan Chen,et al.  Regularized Negative Correlation Learning for Neural Network Ensembles , 2009, IEEE Transactions on Neural Networks.

[34]  Huanhuan Chen,et al.  Model-Based Oversampling for Imbalanced Sequence Classification , 2016, CIKM.

[35]  Huanhuan Chen,et al.  Probabilistic Classification Vector Machines , 2009, IEEE Transactions on Neural Networks.

[36]  Huanhuan Chen,et al.  Multiclass Probabilistic Classification Vector Machine , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.