Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey

Graphs arise naturally in many real-world applications including social networks, recommender systems, ontologies, biology, and computational finance. Traditionally, machine learning models for graphs have been mostly designed for static graphs. However, many applications involve evolving graphs. This introduces important challenges for learning and inference since nodes, attributes, and edges change over time. In this survey, we review the recent advances in representation learning for dynamic graphs, including dynamic knowledge graphs. We describe existing models from an encoder-decoder perspective, categorize these encoders and decoders based on the techniques they employ, and analyze the approaches in each category. We also review several prominent applications and widely used datasets, and highlight directions for future research.

[1]  Heiner Stuckenschmidt,et al.  Marrying Uncertainty and Time in Knowledge Graphs , 2017, AAAI.

[2]  Jian Yang,et al.  Spatio-Temporal Graph Convolution for Skeleton Based Action Recognition , 2018, AAAI.

[3]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[4]  Timothy M. Hospedales,et al.  Hypernetwork Knowledge Graph Embeddings , 2018, ICANN.

[5]  Shih-Chii Liu,et al.  Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences , 2016, NIPS.

[6]  Jeffrey R. Russell,et al.  Autoregressive Conditional Duration: A New Model for Irregularly Spaced Transaction Data , 1998 .

[7]  Alexander J. Smola,et al.  Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning , 2017, ICLR.

[8]  Partha Talukdar,et al.  HyTE: Hyperplane-based Temporally aware Knowledge Graph Embedding , 2018, EMNLP.

[9]  Tianqi Chen,et al.  Net2Net: Accelerating Learning via Knowledge Transfer , 2015, ICLR.

[10]  Jugal K. Kalita,et al.  Abstractive Summarization Using Attentive Neural Techniques , 2018, ArXiv.

[11]  Volker Tresp,et al.  Embedding models for episodic knowledge graphs , 2018, J. Web Semant..

[12]  Hui Li,et al.  On Multi-Relational Link Prediction with Bilinear Models , 2017, AAAI.

[13]  Petros Daras,et al.  evolve2vec: Learning Network Representations Using Temporal Unfolding , 2018, MMM.

[14]  Herbert W. Hethcote,et al.  The Mathematics of Infectious Diseases , 2000, SIAM Rev..

[15]  Nicola De Cao,et al.  Question Answering by Reasoning Across Documents with Graph Convolutional Networks , 2018, NAACL.

[16]  Tore Opsahl.  Triadic closure in two-mode networks: Redefining the global and local clustering coefficients , 2013, Soc. Networks.

[17]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[18]  C. Bilgin.  Dynamic Network Evolution : Models , Clustering , Anomaly Detection , 2009 .

[19]  Steven Skiena,et al.  Don't Walk, Skip!: Online Learning of Multi-scale Network Embeddings , 2016, ASONAM.

[20]  Evangelos E. Papalexakis,et al.  SamBaTen: Sampling-based Batch Incremental Tensor Decomposition , 2017, SDM.

[21]  Yulong Pei,et al.  Node classification in dynamic social networks , 2016 .

[22]  Silvio Savarese,et al.  Structural-RNN: Deep Learning on Spatio-Temporal Graphs , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Xavier Bresson,et al.  Structured Sequence Modeling with Graph Convolutional Recurrent Networks , 2016, ICONIP.

[24]  Bin Li,et al.  Sampling-based algorithm for link prediction in temporal networks , 2016, Inf. Sci..

[25]  Guy Van den Broeck.  On the Complexity and Approximation of Binary Evidence in Lifted Inference , 2013, StarAI@AAAI.

[26]  Le Song,et al.  Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs , 2017, ICML.

[27]  Mohammad Al Hasan,et al.  Link Prediction in Dynamic Networks Using Graphlet , 2016, ECML/PKDD.

[28]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[29]  Charu C. Aggarwal,et al.  Evolutionary Network Analysis , 2014, ACM Comput. Surv..

[30]  W. Weibull.  A Statistical Distribution Function of Wide Applicability , 1951 .

[31]  Manohar Kaul,et al.  Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs , 2019, ACL.

[32]  Heiner Stuckenschmidt,et al.  Rule Based Temporal Inference , 2017, ICLP.

[33]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[34]  Jennifer Neville,et al.  Temporal-Relational Classifiers for Prediction in Evolving Domains , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[35]  Phi Vu Tran,et al.  Learning to Make Predictions on Graphs with Autoencoders , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[36]  Carey E. Priebe,et al.  Out-of-sample extension of graph adjacency spectral embedding , 2018, ICML.

[37]  Aijun An,et al.  dynnode2vec: Scalable Dynamic Network Embedding , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[38]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[39]  Charu C. Aggarwal,et al.  When will it happen?: relationship prediction in heterogeneous information networks , 2012, WSDM '12.

[40]  Charu C. Aggarwal,et al.  Evolutionary Clustering and Analysis of Bibliographic Networks , 2011, 2011 International Conference on Advances in Social Networks Analysis and Mining.

[41]  Ling Chen,et al.  An efficient algorithm for link prediction in temporal uncertain social networks , 2016, Inf. Sci..

[42]  Jure Leskovec,et al.  Community Interaction and Conflict on the Web , 2018, WWW.

[43]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[44]  M. Brand,et al.  Fast low-rank modifications of the thin singular value decomposition , 2006 .

[45]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[46]  Alessandro Rozza,et al.  Dynamic Graph Convolutional Networks , 2017, Pattern Recognit..

[47]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[48]  Stephan Mandt,et al.  Dynamic Word Embeddings , 2017, ICML.

[49]  Palash Goyal,et al.  dyngraph2vec: Capturing Network Dynamics using Dynamic Graph Representation Learning , 2018, Knowl. Based Syst..

[50]  Chris Chatfield,et al.  Holt‐Winters Forecasting: Some Practical Issues , 1988 .

[51]  Jinyin Chen,et al.  GC-LSTM: Graph Convolution Embedded LSTM for Dynamic Link Prediction , 2018, ArXiv.

[52]  Mathias Niepert,et al.  Learning Sequence Encoders for Temporal Knowledge Graph Completion , 2018, EMNLP.

[53]  Zheng Zhang,et al.  MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems , 2015, ArXiv.

[54]  H. Stuckenschmidt,et al.  Applying Markov Logic for Debugging Probabilistic Temporal Knowledge Bases , 2014 .

[55]  Song Bai,et al.  Hypergraph Convolution and Hypergraph Attention , 2019, Pattern Recognit..

[56]  Qing Xie,et al.  A Hybrid Time-Series Link Prediction Framework for Large Social Network , 2012, DEXA.

[57]  Siamak Ravanbakhsh,et al.  Improved Knowledge Graph Embedding using Background Taxonomic Information , 2018, AAAI.

[58]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[59]  Jens Lehmann,et al.  Temporal Knowledge Graph Embedding Model based on Additive Time Series Decomposition , 2019, ArXiv.

[60]  Zhiqiang Xie,et al.  An adaptive random walk sampling method on dynamic community detection , 2016, Expert Syst. Appl..

[61]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[62]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.

[63]  Carey E. Priebe,et al.  A Consistent Adjacency Spectral Embedding for Stochastic Blockmodel Graphs , 2011, 1108.2228.

[64]  Le Song,et al.  Deep Coevolutionary Network: Embedding User and Item Features for Recommendation , 2016, 1609.03675.

[65]  Volker Tresp,et al.  Embedding Learning for Declarative Memories , 2017, ESWC.

[66]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[67]  Heinrich Müller,et al.  SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[68]  Jian Zhang,et al.  A Survey on Streaming Algorithms for Massive Graphs , 2010, Managing and Mining Graph Data.

[69]  Andrew B. Whinston,et al.  Path to Purchase: A Mutually Exciting Point Process Model for Online Advertising and Conversion , 2012, Manag. Sci..

[70]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.

[71]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[72]  Ni Lao,et al.  Relational retrieval using a combination of path-constrained random walks , 2010, Machine Learning.

[73]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.

[74]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[75]  Alex Pentland,et al.  Sensing the "Health State" of a Community , 2012, IEEE Pervasive Computing.

[76]  Yue Gao,et al.  Hypergraph Neural Networks , 2018, AAAI.

[77]  William Brendel,et al.  Learning spatiotemporal graphs of human activities , 2011, 2011 International Conference on Computer Vision.

[78]  Jure Leskovec,et al.  Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change , 2016, ACL.

[79]  Przemyslaw Kazienko,et al.  Matching Organizational Structure and Social Network Extracted from Email Communication , 2011, BIS.

[80]  Hanghang Tong,et al.  Fast Eigen-Functions Tracking on Dynamic Graphs , 2015, SDM.

[81]  Vineeth N. Balasubramanian,et al.  STwalk: learning trajectory representations in temporal graphs , 2017, COMAD/CODS.

[82]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[83]  Nicolas Usunier,et al.  Canonical Tensor Decomposition for Knowledge Base Completion , 2018, ICML.

[84]  Apurva Narayan,et al.  Learning Graph Dynamics using Deep Neural Networks , 2018 .

[85]  Jon M. Kleinberg,et al.  Overview of the 2003 KDD Cup , 2003, SKDD.

[86]  Ricardo B. C. Prudêncio,et al.  Time Series Based Link Prediction , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[87]  Guillaume Bouchard,et al.  Knowledge Graph Completion via Complex Tensor Factorization , 2017, J. Mach. Learn. Res..

[88]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[89]  Chengqi Zhang,et al.  MetaGraph2Vec: Complex Semantic Path Augmented Heterogeneous Network Embedding , 2018, PAKDD.

[90]  Ali Farhadi,et al.  Bidirectional Attention Flow for Machine Comprehension , 2016, ICLR.

[91]  Jian Pei,et al.  Asymmetric Transitivity Preserving Graph Embedding , 2016, KDD.

[92]  Ryohei Hisano,et al.  Semi-supervised Graph Embedding Approach to Dynamic Link Prediction , 2016, ArXiv.

[93]  Steven Skiena,et al.  Walklets: Multiscale Graph Embeddings for Interpretable Network Classification , 2016, ArXiv.

[94]  Pedro A. Szekely,et al.  Recurrent Event Network for Reasoning over Temporal Knowledge Graphs , 2019, ArXiv.

[95]  Heiko Paulheim,et al.  RDF2Vec: RDF Graph Embeddings for Data Mining , 2016, SEMWEB.

[96]  Jure Leskovec,et al.  Learning Dynamic Embeddings from Temporal Interactions , 2018, ArXiv.

[97]  Jiawei Han,et al.  A Particle-and-Density Based Evolutionary Clustering Method for Dynamic Networks , 2009, Proc. VLDB Endow..

[98]  Zehra Cataltepe,et al.  Link prediction using time series of neighborhood-based node similarity scores , 2015, Data Mining and Knowledge Discovery.

[99]  Jiliang Tang,et al.  Streaming Graph Neural Networks , 2018, SIGIR.

[100]  A. Hawkes.  Spectra of some self-exciting and mutually exciting point processes , 1971 .

[101]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[102]  Larry S. Davis,et al.  Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmentation , 2018, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[103]  J. L. Roux.  An Introduction to the Kalman Filter , 2003 .

[104]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[105]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[106]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[107]  David Poole,et al.  Knowledge Hypergraphs: Prediction Beyond Binary Relations , 2019, IJCAI.

[108]  Gerard Delanty.  The Foundations of Social Theory , 2009 .

[109]  Jian Pei,et al.  TIMERS: Error-Bounded SVD Restart on Dynamic Networks , 2017, AAAI.

[110]  Zhanxing Zhu,et al.  3D Graph Convolutional Networks with Temporal Graphs: A Spatial Information Free Framework For Traffic Forecasting , 2019, ArXiv.

[111]  Qi Xuan,et al.  E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[112]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[113]  Ryan A. Rossi,et al.  The Network Data Repository with Interactive Graph Analytics and Visualization , 2015, AAAI.

[114]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[115]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[116]  Igor Jurisica,et al.  Modeling interactome: scale-free or geometric? , 2004, Bioinform..

[117]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[118]  Wenwu Zhu,et al.  Structural Deep Network Embedding , 2016, KDD.

[119]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[120]  Zhifang Sui,et al.  Towards Time-Aware Knowledge Graph Completion , 2016, COLING.

[121]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[122]  Arijit Khan,et al.  Efficiently Embedding Dynamic Knowledge Graphs , 2019, ArXiv.

[123]  Stephan Günnemann,et al.  Introduction to Tensor Decompositions and their Applications in Machine Learning , 2017, ArXiv.

[124]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[125]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[126]  Ryan A. Rossi,et al.  Dynamic Network Embeddings: From Random Walks to Temporal Random Walks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[127]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[128]  Srijan Kumar,et al.  Learning Dynamic Embeddings from Temporal Interaction Networks , 2018 .

[129]  Zhizhen Zhao,et al.  LanczosNet: Multi-Scale Deep Graph Convolutional Networks , 2019, ICLR.

[130]  Julien Leblay,et al.  Deriving Validity Time in Knowledge Graph , 2018, WWW.

[131]  Alex Beutel,et al.  Recurrent Recommender Networks , 2017, WSDM.

[132]  Volker Tresp,et al.  Predicting the co-evolution of event and Knowledge Graphs , 2015, 2016 19th International Conference on Information Fusion (FUSION).

[133]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[134]  V. Isham,et al.  A self-correcting point process , 1979 .

[135]  Chandler Davis.  The rotation of eigenvectors by a perturbation , 1963 .

[136]  Myra Spiliopoulou,et al.  Evolution in Social Networks: A Survey , 2011, Social Network Data Analytics.

[137]  Ryan A. Rossi,et al.  Continuous-Time Dynamic Network Embeddings , 2018, WWW.

[138]  P. Stange.  On the efficient update of the Singular Value Decomposition , 2008 .

[139]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[140]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[141]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[142]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[143]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[144]  Zan Huang,et al.  The Time-Series Link Prediction Problem with Applications in Communication Surveillance , 2009, INFORMS J. Comput..

[145]  Tom M. Mitchell,et al.  Random Walk Inference and Learning in A Large Scale Knowledge Base , 2011, EMNLP.

[146]  Jia Li,et al.  Latent Cross: Making Use of Context in Recurrent Recommender Systems , 2018, WSDM.

[147]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[148]  Abhinav Gupta,et al.  Videos as Space-Time Region Graphs , 2018, ECCV.

[149]  Geoffrey Zweig,et al.  Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[150]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[151]  Pascal Poupart,et al.  Diachronic Embedding for Temporal Knowledge Graph Completion , 2019, AAAI.

[152]  Utkarsh Upadhyay,et al.  Recurrent Marked Temporal Point Processes: Embedding Event History to Vector , 2016, KDD.

[153]  Jason Eisner,et al.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process , 2016, NIPS.

[154]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[155]  Ling Chen,et al.  Link prediction in dynamic social networks by integrating different types of information , 2014, Applied Intelligence.

[156]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[157]  Jingyuan Zhang,et al.  Knowledge Graph Embedding Based Question Answering , 2019, WSDM.

[158]  Seyed Mehran Kazemi.  Representing and learning relations and properties under uncertainty , 2018 .

[159]  Steven Schockaert,et al.  Lifted Relational Neural Networks: Efficient Learning of Latent Relational Structures , 2018, J. Artif. Intell. Res..

[160]  Tamara G. Kolda,et al.  Temporal Link Prediction Using Matrix and Tensor Factorizations , 2010, TKDD.

[161]  Junjie Wu,et al.  Embedding Temporal Network via Neighborhood Formation , 2018, KDD.

[162]  Jie Chen,et al.  EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs , 2020, AAAI.

[163]  S. V. N. Vishwanathan,et al.  A Structural Smoothing Framework For Robust Graph Comparison , 2015, NIPS.

[164]  Rui Chen,et al.  Real-Time Streaming Graph Embedding Through Local Actions , 2019, WWW.

[165]  Ido Guy,et al.  Node Embedding over Temporal Graphs , 2019, IJCAI.

[166]  Tong Wang,et al.  Link Prediction in Evolving Networks Based on Popularity of Nodes , 2017, Scientific Reports.

[167]  Kristian Kersting,et al.  Relational Logistic Regression , 2014, KR.

[168]  Henry A. Kautz,et al.  Slice Normalized Dynamic Markov Logic Networks , 2012, NIPS.

[169]  Henry A. Kautz,et al.  Recognizing Multi-Agent Activities from GPS Data , 2010, AAAI.

[170]  Luc De Raedt,et al.  Statistical Relational Artificial Intelligence: Logic, Probability, and Computation , 2016, Statistical Relational Artificial Intelligence.

[171]  Hao Hu,et al.  State-Frequency Memory Recurrent Neural Networks , 2017, ICML.

[172]  Yuriy Tyshetskiy,et al.  Efficient Representation Learning Using Random Walks for Dynamic Graphs , 2019, ArXiv.

[173]  Wenwu Zhu,et al.  DepthLGP: Learning Embeddings of Out-of-Sample Nodes in Dynamic Networks , 2018, AAAI.

[174]  Guy Van den Broeck,et al.  Lifted generative learning of Markov logic networks , 2016, Machine Learning.

[175]  Purnamrita Sarkar,et al.  A Latent Space Approach to Dynamic Embedding of Co-occurrence Data , 2007, AISTATS.

[176]  Prateek Yadav,et al.  HyperGCN: Hypergraph Convolutional Networks for Semi-Supervised Classification , 2018, ArXiv.

[177]  Alexander Peysakhovich,et al.  PyTorch-BigGraph: A Large-scale Graph Embedding System , 2019, SysML.

[178]  Christophe Diot,et al.  Impact of Human Mobility on Opportunistic Forwarding Algorithms , 2007, IEEE Transactions on Mobile Computing.

[179]  Charu C. Aggarwal,et al.  NetWalk: A Flexible Deep Embedding Approach for Anomaly Detection in Dynamic Networks , 2018, KDD.

[180]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[181]  Sanjay Thakur,et al.  Time2Vec: Learning a Vector Representation of Time , 2019, ArXiv.

[182]  Jang-Gyu Lee,et al.  On updating the singular value decomposition , 1996, Proceedings of International Conference on Communication Technology. ICCT '96.

[183]  Samy Bengio,et al.  Time-Dependent Representation for Neural Event Sequence Prediction , 2017, ICLR.

[184]  Marc Peter Deisenroth,et al.  Neural Embeddings of Graphs in Hyperbolic Space , 2017, ArXiv.

[185]  Seyed Mehran Kazemi,et al.  SimplE Embedding for Link Prediction in Knowledge Graphs , 2018, NeurIPS.

[186]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[187]  Philip S. Yu,et al.  Heterogeneous Information Network Embedding for Recommendation , 2017, IEEE Transactions on Knowledge and Data Engineering.

[188]  Bart Baesens,et al.  Combining Temporal Aspects of Dynamic Networks with Node2Vec for a more Efficient Dynamic Link Prediction , 2018, 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM).

[189]  Derek Bridge,et al.  Diversity, Serendipity, Novelty, and Coverage , 2016, ACM Trans. Interact. Intell. Syst..

[190]  Hung Hai Bui,et al.  Automorphism Groups of Graphical Models and Lifted Variational Inference , 2012, UAI.

[191]  Seyed Mehran Kazemi,et al.  Bridging Weighted Rules and Graph Random Walks for Statistical Relational Models , 2018, Front. Robot. AI.

[192]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[193]  Jessika Schulze,et al.  Handbook Of Logic In Artificial Intelligence And Logic Programming , 2016 .

[194]  Masashi Shimbo,et al.  On the Equivalence of Holographic and Complex Embeddings for Link Prediction , 2017, ACL.

[195]  Yiming Yang,et al.  Introducing the Enron Corpus , 2004, CEAS.

[196]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[197]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[198]  Qian Xu,et al.  Overlapping Community Detection Based on Random Walk and Seeds Extension , 2017, ChineseCSCW.

[199]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[200]  Joan Bruna,et al.  On the equivalence between graph isomorphism testing and function approximation with GNNs , 2019, NeurIPS.

[201]  P. Dobson,et al.  Distinguishing enzyme structures from non-enzymes without alignments. , 2003, Journal of molecular biology.

[202]  Mohammad Al Hasan,et al.  A Survey of Link Prediction in Social Networks , 2011, Social Network Data Analytics.

[203]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[204]  Nikos Komodakis,et al.  Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[205]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[206]  Lizhen Qu,et al.  STransE: a novel embedding model of entities and relationships in knowledge bases , 2016, NAACL.

[207]  Yun Chi,et al.  On evolutionary spectral clustering , 2009, TKDD.

[208]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[209]  Gerhard Weikum,et al.  YAGO2: A Spatially and Temporally Enhanced Knowledge Base from Wikipedia: Extended Abstract , 2013, IJCAI.

[210]  Ce Zhang,et al.  DeepDive: A Data Management System for Automatic Knowledge Base Construction , 2015 .

[211]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[212]  Deepayan Chakrabarti,et al.  Evolutionary clustering , 2006, KDD '06.

[213]  Yaron Lipman,et al.  On the Universality of Invariant Networks , 2019, ICML.

[214]  Jure Leskovec,et al.  Embedding Logical Queries on Knowledge Graphs , 2018, NeurIPS.

[215]  Rajarshi Das,et al.  Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension , 2018, ICLR.

[216]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[217]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[218]  Ashish Vaswani,et al.  Self-Attention with Relative Position Representations , 2018, NAACL.

[219]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[220]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[221]  O. Aalen,et al.  Survival and Event History Analysis: A Process Point of View , 2008 .

[222]  Volker Tresp,et al.  Learning with Memory Embeddings , 2015, ArXiv.

[223]  Kristian Kersting,et al.  Counting Belief Propagation , 2009, UAI.

[224]  László Babai,et al.  Canonical labelling of graphs in linear average time , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[225]  Guy Van den Broeck,et al.  New Liftable Classes for First-Order Probabilistic Inference , 2016, NIPS.

[226]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[227]  Yueting Zhuang,et al.  Dynamic Network Embedding by Modeling Triadic Closure Process , 2018, AAAI.

[228]  Huan Liu,et al.  Relational learning via latent social dimensions , 2009, KDD.

[229]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[230]  Philip S. Yu,et al.  A Survey of Heterogeneous Information Network Analysis , 2015, IEEE Transactions on Knowledge and Data Engineering.

[231]  Hans-Peter Kriegel,et al.  Protein function prediction via graph kernels , 2005, ISMB.

[232]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[233]  E. Bacry,et al.  Hawkes Processes in Finance , 2015, 1502.04592.

[234]  Muthu Manikandan Baskaran,et al.  Computationally Efficient CP Tensor Decomposition Update Framework for Emerging Component Discovery in Streaming Data , 2018, 2018 IEEE High Performance extreme Computing Conference (HPEC).

[235]  Xavier Bresson,et al.  CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters , 2017, IEEE Transactions on Signal Processing.

[236]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[237]  Chengqi Zhang,et al.  Network Representation Learning: A Survey , 2017, IEEE Transactions on Big Data.

[238]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[239]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[240]  Yizhou Sun,et al.  Mining heterogeneous information networks: a structural analysis approach , 2013, SKDD.

[241]  Deng Cai,et al.  What to Do Next: Modeling User Behaviors by Time-LSTM , 2017, IJCAI.

[242]  P. A. W. Lewis,et al.  Multivariate point processes , 2018, Point Processes.

[243]  Xiaoming Fu,et al.  Triadic Closure Pattern Analysis and Prediction in Social Networks , 2015, IEEE Transactions on Knowledge and Data Engineering.

[244]  Aynaz Taheri,et al.  Learning to Represent the Evolution of Dynamic Graphs with Recurrent Models , 2019, WWW.

[245]  Mohammad Reza Meybodi,et al.  A novel time series link prediction method: Learning automata approach , 2017 .

[246]  H. Akaike.  Fitting autoregressive models for prediction , 1969 .

[247]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[248]  Joan Bruna,et al.  Community Detection with Graph Neural Networks , 2017, 1705.08415.

[249]  Natasa Przulj,et al.  Biological network comparison using graphlet degree distribution , 2007, Bioinform..

[250]  Seyed Mehran Kazemi,et al.  RelNN: A Deep Neural Model for Relational Learning , 2017, AAAI.

[251]  Ciro Cattuto,et al.  What's in a crowd? Analysis of face-to-face behavioral networks , 2010, Journal of theoretical biology.

[252]  Gerhard Weikum,et al.  TEQUILA: Temporal Question Answering over Knowledge Bases , 2018, CIKM.

[253]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[254]  Wei Zhang,et al.  Dynamic Graph Representation Learning via Self-Attention Networks , 2018, ArXiv.

[255]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[256]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[257]  Amin Milani Fard,et al.  Relationship Prediction in Dynamic Heterogeneous Information Networks , 2019, ECIR.

[258]  Neil Immerman,et al.  An optimal lower bound on the number of variables for graph identification , 1992, Comb..

[259]  Hongyuan Zha,et al.  DyRep: Learning Representations over Dynamic Graphs , 2019, ICLR.

[260]  Gabriel Peyré,et al.  Universal Invariant and Equivariant Graph Neural Networks , 2019, NeurIPS.

[261]  Lise Getoor,et al.  A short introduction to probabilistic soft logic , 2012, NIPS 2012.

[262]  Andrew McCallum,et al.  Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data , 2004, J. Mach. Learn. Res..

[263]  Le Song,et al.  Learning Steady-States of Iterative Algorithms over Graphs , 2018, ICML.

[264]  Xi Chen,et al.  Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization , 2010, SDM.

[265]  Guojie Song,et al.  Dynamic Network Embedding : An Extended Approach for Skip-gram based Network Embedding , 2018, IJCAI.

[266]  Charu C. Aggarwal,et al.  Link Prediction with Spatial and Temporal Consistency in Dynamic Networks , 2017, IJCAI.

[267]  Huan Liu,et al.  Attributed Network Embedding for Learning in a Dynamic Environment , 2017, CIKM.

[268]  A. Stephen McGough,et al.  Temporal Graph Offset Reconstruction: Towards Temporally Robust Graph Representation Learning , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[269]  Yiming Yang,et al.  Analogical Inference for Multi-relational Embeddings , 2017, ICML.

[270]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[271]  Jianxin Li,et al.  On the Representation and Embedding of Knowledge Bases beyond Binary Relations , 2016, IJCAI.

[272]  Jérôme Kunegis,et al.  KONECT: the Koblenz network collection , 2013, WWW.

[273]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[274]  Martin Theobald,et al.  A Temporal-Probabilistic Database Model for Information Extraction , 2013, Proc. VLDB Endow..

[275]  Hao Ma,et al.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs , 2018, UAI.

[276]  Guy Van den Broeck.  On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference , 2011, NIPS.

[277]  Jon M. Kleinberg,et al.  The link-prediction problem for social networks , 2007, J. Assoc. Inf. Sci. Technol..

[278]  Lin Yao,et al.  The 7 th International Conference on Ambient Systems , Networks and Technologies ( ANT 2016 ) Link Prediction Based on Common-Neighbors for Dynamic Social Network , 2016 .

[279]  Wei Lu,et al.  Deep Neural Networks for Learning Graph Representations , 2016, AAAI.

[280]  Yiqun Liu,et al.  Temporal Relational Ranking for Stock Prediction , 2018, ACM Trans. Inf. Syst..

[281]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[282]  Lukasz Kaiser,et al.  Universal Transformers , 2018, ICLR.