Learning Neural Ordinary Equations for Forecasting Future Links on Temporal Knowledge Graphs

There has been an increasing interest in inferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing approaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neural ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural information into continuous-time dynamic embeddings. In addition, a novel graph transition layer is applied to capture the transitions on the dynamic graph, i.e., edge formation and dissolution. We perform extensive experiments on five benchmark datasets for temporal KG reasoning, showing our model’s superior performance on the future link forecasting task.

[1]  Alexander J. Smola,et al.  Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning , 2017, ICLR.

[2]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[3]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[4]  Timothy M. Hospedales,et al.  TuckER: Tensor Factorization for Knowledge Graph Completion , 2019, EMNLP.

[5]  Mathias Niepert,et al.  Learning Sequence Encoders for Temporal Knowledge Graph Completion , 2018, EMNLP.

[6]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[7]  Huzefa Rangwala,et al.  Dynamic Knowledge Graph based Multi-Event Forecasting , 2020, KDD.

[8]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[9]  Stephan Günnemann,et al.  Graph Hawkes Network for Reasoning on Temporal Knowledge Graphs , 2020, ArXiv.

[10]  Pedro A. Szekely,et al.  Recurrent Event Network : Global Structure Inference Over Temporal Knowledge Graph , 2019 .

[11]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[12]  Volker Tresp,et al.  xERTE: Explainable Reasoning on Temporal Knowledge Graphs for Forecasting Future Links , 2020, ArXiv.

[13]  Pascal Poupart,et al.  Diachronic Embedding for Temporal Knowledge Graph Completion , 2019, AAAI.

[14]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[15]  Fabian M. Suchanek,et al.  YAGO3: A Knowledge Base from Multilingual Wikipedias , 2015, CIDR.

[16]  Julien Leblay,et al.  Deriving Validity Time in Knowledge Graph , 2018, WWW.

[17]  E. Tyrtyshnikov.  A brief introduction to numerical analysis , 1997 .

[18]  Jure Leskovec,et al.  Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs , 2020, NeurIPS.

[19]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[20]  Guangming Shi,et al.  Temporal Graph Modeling for Skeleton-based Action Recognition , 2020, ArXiv.

[21]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[22]  Alexandr Katrutsa,et al.  Interpolated Adjoint Method for Neural ODEs , 2020, ArXiv.

[23]  Lloyd N. Trefethen,et al.  Barycentric Lagrange Interpolation , 2004, SIAM Rev..

[24]  Changjun Fan,et al.  Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks , 2021, AAAI.

[25]  Le Song,et al.  Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs , 2017, ICML.

[26]  Nicolas Usunier,et al.  Tensor Decompositions for temporal knowledge base completion , 2020, ICLR.