E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction

Predicting the potential relations between nodes in networks, known as link prediction, has long been a challenge in network science. However, most studies just focused on link prediction of static network, while real-world networks always evolve over time with the occurrence and vanishing of nodes and links. Dynamic network link prediction (DNLP) thus has been attracting more and more attention since it can better capture the evolution nature of networks, but still most algorithms fail to achieve satisfied prediction accuracy. Motivated by the excellent performance of long short-term memory (LSTM) in processing time series, in this article, we propose a novel encoder-LSTM-decoder (E-LSTM-D) deep learning model to predict dynamic links end to end. It could handle long-term prediction problems, and suits the networks of different scales with fine-tuned structure. To the best of our knowledge, it is the first time that LSTM, together with an encoder–decoder architecture, is applied to link prediction in dynamic networks. This new model is able to automatically learn structural and temporal features in a unified framework, which can predict the links that never appear in the network before. The extensive experiments show that our E-LSTM-D model significantly outperforms newly proposed DNLP methods and obtain the state-of-the-art results.

[1]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[2]  M. Newman Clustering and preferential attachment in growing networks. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  M. Brand,et al.  Fast low-rank modifications of the thin singular value decomposition , 2006 .

[4]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[5]  Linyuan Lü,et al.  Predicting missing links via local information , 2009, 0901.0553.

[6]  Anton J. Enright,et al.  Network visualization and analysis of gene expression data using BioLayout Express3D , 2009, Nature Protocols.

[7]  Zan Huang,et al.  The Time-Series Link Prediction Problem with Applications in Communication Surveillance , 2009, INFORMS J. Comput..

[8]  David A. Bader,et al.  Massive Social Network Analysis: Mining Twitter for Social Good , 2010, 2010 39th International Conference on Parallel Processing.

[9]  Linyuan Lu,et al.  Link Prediction in Complex Networks: A Survey , 2010, ArXiv.

[10]  Przemyslaw Kazienko,et al.  Matching Organizational Structure and Social Network Extracted from Email Communication , 2011, BIS.

[11]  Yang Xiao,et al.  A survey of communication/networking in Smart Grids , 2012, Future Gener. Comput. Syst..

[12]  Nitesh V. Chawla,et al.  Evaluating link prediction methods , 2014, Knowledge and Information Systems.

[13]  Ling Chen,et al.  Link prediction in dynamic social networks by integrating different types of information , 2014, Applied Intelligence.

[14]  Razvan Pascanu,et al.  How to Construct Deep Recurrent Neural Networks , 2013, ICLR.

[15]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[16]  Hui Li,et al.  A Deep Learning Approach to Link Prediction in Dynamic Networks , 2014, SDM.

[17]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[18]  Maman Abdurachman Djauhari,et al.  Correlation network analysis for multi-dimensional data in stocks market , 2015 .

[19]  Qi Xuan,et al.  Temporal motifs reveal collaboration patterns in online task-oriented networks. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[20]  Ryan A. Rossi,et al.  The Network Data Repository with Interactive Graph Analytics and Visualization , 2015, AAAI.

[21]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[22]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[23]  Philipp Hövel,et al.  Disease Spread through Animal Movements: A Static and Temporal Network Analysis of Pig Trade in Germany , 2016, PloS one.

[24]  Lin Yao,et al.  The 7 th International Conference on Ambient Systems , Networks and Technologies ( ANT 2016 ) Link Prediction Based on Common-Neighbors for Dynamic Social Network , 2016 .

[25]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[26]  Scalable Temporal Latent Space Inference for Link Prediction in Dynamic Social Networks , 2017, IEEE Transactions on Knowledge and Data Engineering.

[27]  Vijay K. Devabhaktuni,et al.  Evaluating Link Prediction Accuracy in Dynamic Networks with Added and Removed Edges , 2016, 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom).

[28]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[29]  Ling Chen,et al.  An efficient algorithm for link prediction in temporal uncertain social networks , 2016, Inf. Sci..

[30]  Bin Li,et al.  Sampling-based algorithm for link prediction in temporal networks , 2016, Inf. Sci..

[31]  Li Sun,et al.  Efficient incremental dynamic link prediction algorithms in social network , 2017, Knowl. Based Syst..

[32]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[33]  Alper Ozcan,et al.  Link prediction in evolving heterogeneous networks using the NARX neural networks , 2018, Knowledge and Information Systems.

[34]  Qi Xuan,et al.  Pinning control of clustered complex networks with different size , 2017 .

[35]  Qi Xuan,et al.  Link Weight Prediction Using Supervised Learning Methods and Its Application to Yelp Layered Network , 2018, IEEE Transactions on Knowledge and Data Engineering.

[36]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[37]  Jian Pei,et al.  TIMERS: Error-Bounded SVD Restart on Dynamic Networks , 2017, AAAI.

[38]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.

[39]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[40]  Yonghong Yan,et al.  Restricted Boltzmann Machine-Based Approaches for Link Prediction in Dynamic Networks , 2018, IEEE Access.

[41]  Qi Xuan,et al.  Evolving Convolutional Neural Network and Its Application in Fine-Grained Visual Categorization , 2018, IEEE Access.

[42]  Jian Zhang,et al.  Automatic Pearl Classification Machine Based on a Multistream Convolutional Neural Network , 2018, IEEE Transactions on Industrial Electronics.

[43]  Qi Xuan,et al.  Modern Food Foraging Patterns: Geography and Cuisine Choices of Restaurant Patrons on Yelp , 2018, IEEE Transactions on Computational Social Systems.

[44]  Philip S. Yu,et al.  Deep Dynamic Network Embedding for Link Prediction , 2018, IEEE Access.

[45]  Charu C. Aggarwal,et al.  NetWalk: A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks , 2018, KDD.

[46]  Qi Xuan,et al.  Social Synchrony on Complex Networks , 2018, IEEE Transactions on Cybernetics.

[47]  Yueting Zhuang,et al.  Dynamic Network Embedding by Modeling Triadic Closure Process , 2018, AAAI.

[48]  Yu Wang,et al.  Graph regularized nonnegative matrix factorization for temporal link prediction in dynamic networks , 2018 .

[49]  Junwei Han,et al.  Deep Spatiality: Unsupervised Learning of Spatially-Enhanced Global and Local 3D Features by Deep Neural Network With Coupled Softmax , 2018, IEEE Transactions on Image Processing.

[50]  Junwei Han,et al.  SeqViews2SeqLabels: Learning 3D Global Features via Aggregating Sequential Views by RNN With Attention , 2019, IEEE Transactions on Image Processing.

[51]  Jianguo Luo,et al.  Complex Network Construction of Multivariate Time Series Using Information Geometry , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[52]  Alper Ozcan,et al.  Multivariate Time Series Link Prediction for Evolving Heterogeneous Network , 2019, Int. J. Inf. Technol. Decis. Mak..

[53]  Lin Wang,et al.  Investigating the Evolution of a Neuroplasticity Network for Learning , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[54]  Zhigang Zeng,et al.  Optimizing Pinning Control of Complex Dynamical Networks Based on Spectral Properties of Grounded Laplacian Matrices , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.