Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs

The availability of large scale event data with time stamps has given rise to dynamically evolving knowledge graphs that contain temporal information for each edge. Reasoning over time in such dynamic knowledge graphs is not yet well understood. To this end, we present Know-Evolve, a novel deep evolutionary knowledge network that learns non-linearly evolving entity representations over time. The occurrence of a fact (edge) is modeled as a multivariate point process whose intensity function is modulated by the score for that fact computed based on the learned entity embeddings. We demonstrate significantly improved performance over various relational learning approaches on two large scale real-world datasets. Further, our method effectively predicts occurrence or recurrence time of a fact which is novel compared to prior reasoning approaches in multi-relational setting.

[1]  Michelangelo Ceci,et al.  Relational mining for discovering changes in evolving networks , 2015, Neurocomputing.

[2]  Le Song,et al.  Discriminative Embeddings of Latent Variable Models for Structured Data , 2016, ICML.

[3]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[4]  Zhifang Sui,et al.  Encoding Temporal Information for Time-Aware Link Prediction , 2016, EMNLP.

[5]  Le Song,et al.  Time-Sensitive Recommendation From Recurrent User Activities , 2015, NIPS.

[6]  Le Song,et al.  A Stochastic Differential Equation Framework for Guiding Online User Activities in Closed Loop , 2016, AISTATS.

[7]  Yu Hu,et al.  Probabilistic Reasoning via Deep Learning: Neural Association Models , 2016, ArXiv.

[8]  P. A. W. Lewis,et al.  Multivariate point processes , 2018, Point Processes.

[9]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[10]  Wei Zhang,et al.  Knowledge vault: a web-scale approach to probabilistic knowledge fusion , 2014, KDD.

[11]  Le Song,et al.  Steering Opinion Dynamics in Information Diffusion Networks , 2016, ArXiv.

[12]  Corrado Loglisci,et al.  Leveraging temporal autocorrelation of historical data for improving accuracy in network regression , 2017, Stat. Anal. Data Min..

[13]  David M. Blei,et al.  Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations , 2016, ICML.

[14]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[15]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[16]  Utkarsh Upadhyay,et al.  Recurrent Marked Temporal Point Processes: Embedding Event History to Vector , 2016, KDD.

[17]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[18]  A. Hawkes.  Spectra of some self-exciting and mutually exciting point processes , 1971 .

[19]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[20]  P. Grambsch.  Survival and Event History Analysis: A Process Point of View by AALEN, O. O., BORGAN, O., and GJESSING, H. K. , 2009 .

[21]  Volker Tresp,et al.  Predicting the co-evolution of event and Knowledge Graphs , 2015, 2016 19th International Conference on Information Fusion (FUSION).

[22]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[23]  V. Isham,et al.  A self-correcting point process , 1979 .

[24]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[25]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[26]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[27]  Le Song,et al.  Deep Coevolutionary Network: Embedding User and Item Features for Recommendation , 2016, 1609.03675.

[28]  Le Song,et al.  Linking Micro Event History to Macro Prediction in Point Process Models , 2017, AISTATS.

[29]  Danqi Chen,et al.  Observed versus latent features for knowledge base and text inference , 2015, CVSC.

[30]  Le Song,et al.  Coevolutionary Latent Feature Processes for Continuous-Time User-Item Interactions , 2016, NIPS.

[31]  Aapo Hyvärinen,et al.  Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics , 2012, J. Mach. Learn. Res..

[32]  Le Song,et al.  Isotonic Hawkes Processes , 2016, ICML.

[33]  Le Song,et al.  Shaping Social Activity by Incentivizing Users , 2014, NIPS.

[34]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[35]  Jennifer Neville,et al.  Temporal-Relational Classifiers for Prediction in Evolving Domains , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[36]  Shuang Li,et al.  COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution , 2015, NIPS.

[37]  Le Song,et al.  Variational Policy for Guiding Point Processes , 2017, ICML.

[38]  Shuang-Hong Yang,et al.  Mixture of Mutually Exciting Processes for Viral Diffusion , 2013, ICML.