Graph Hawkes Network for Reasoning on Temporal Knowledge Graphs

The Hawkes process has become a standard method for modeling self-exciting event sequences with different event types. A recent work generalizing the Hawkes process to a neurally self-modulating multivariate point process enables the capturing of more complex and realistic influences of past events on the future. However, this approach is limited by the number of event types, making it impossible to model the dynamics of evolving graph sequences, where each possible link between two nodes can be considered as an event type. The problem becomes even more dramatic when links are directional and labeled, since, in this case, the number of event types scales with the number of nodes and link types. To address this issue, we propose the Graph Hawkes Network to capture the dynamics of evolving graph sequences. Extensive experiments on large-scale temporal relational databases, such as temporal knowledge graphs, demonstrate the effectiveness of our approach.

[1]  Jens Lehmann,et al.  Temporal Knowledge Graph Embedding Model based on Additive Time Series Decomposition , 2019, ArXiv.

[2]  Volker Tresp,et al.  Predicting the co-evolution of event and Knowledge Graphs , 2015, 2016 19th International Conference on Information Fusion (FUSION).

[3]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[4]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[5]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[6]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[7]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[8]  Zhifang Sui,et al.  Towards Time-Aware Knowledge Graph Completion , 2016, COLING.

[9]  Partha Talukdar,et al.  HyTE: Hyperplane-based Temporally aware Knowledge Graph Embedding , 2018, EMNLP.

[10]  Sanjay Thakur,et al.  Time2Vec: Learning a Vector Representation of Time , 2019, ArXiv.

[11]  Utkarsh Upadhyay,et al.  Recurrent Marked Temporal Point Processes: Embedding Event History to Vector , 2016, KDD.

[12]  Jason Eisner,et al.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process , 2016, NIPS.

[13]  Linyuan Lu,et al.  Link prediction based on local random walk , 2010, 1001.2467.

[14]  Esko Valkeila,et al.  An Introduction to the Theory of Point Processes, Volume II: General Theory and Structure, 2nd Edition by Daryl J. Daley, David Vere‐Jones , 2008 .

[15]  Pedro A. Szekely,et al.  Recurrent Event Network for Reasoning over Temporal Knowledge Graphs , 2019, ArXiv.

[16]  Volker Tresp,et al.  Embedding models for episodic knowledge graphs , 2018, J. Web Semant..

[17]  Guillaume Bouchard,et al.  Knowledge Graph Completion via Complex Tensor Factorization , 2017, J. Mach. Learn. Res..

[18]  Mathias Niepert,et al.  Learning Sequence Encoders for Temporal Knowledge Graph Completion , 2018, EMNLP.

[19]  Y. Ogata.  Space-Time Point-Process Models for Earthquake Occurrences , 1998 .

[20]  Le Song,et al.  Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes , 2013, AISTATS.

[21]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[22]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[23]  Wei Zhang,et al.  Dynamic Graph Representation Learning via Self-Attention Networks , 2018, ArXiv.

[24]  Le Song,et al.  Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs , 2017, ICML.

[25]  A. Hawkes.  Spectra of some self-exciting and mutually exciting point processes , 1971 .

[26]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[27]  Nicola Fanizzi,et al.  Learning to Propagate Knowledge in Web Ontologies , 2014, URSW.

[28]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[29]  Julien Leblay,et al.  Deriving Validity Time in Knowledge Graph , 2018, WWW.

[30]  Hongyuan Zha,et al.  DyRep: Learning Representations over Dynamic Graphs , 2019, ICLR.

[31]  David M. Blei,et al.  Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations , 2016, ICML.

[32]  E. Bacry,et al.  Estimation of slowly decreasing Hawkes kernels: application to high-frequency order book dynamics , 2016 .

[33]  O. Aalen,et al.  Survival and Event History Analysis: A Process Point of View , 2008 .

[34]  Pascal Poupart,et al.  Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey , 2019, ArXiv.