Temporal Attribute Prediction via Joint Modeling of Multi-Relational Structure Evolution

Time series prediction is an important problem in machine learning. Previous methods for time series prediction did not involve additional information. With a lot of dynamic knowledge graphs available, we can use this additional information to predict the time series better. Recently, there has been a focus on the application of deep representation learning on dynamic graphs. These methods predict the structure of the graph by reasoning over the interactions in the graph at previous time steps. In this paper, we propose a new framework to incorporate the information from dynamic knowledge graphs for time series prediction. We show that if the information contained in the graph and the time series data are closely related, then this inter-dependence can be used to predict the time series with improved accuracy. Our framework, DArtNet, learns a static embedding for every node in the graph as well as a dynamic embedding which is dependent on the dynamic attribute value (time-series). Then it captures the information from the neighborhood by taking a relation specific mean and encodes the history information using RNN. We jointly train the model link prediction and attribute prediction. We evaluate our method on five specially curated datasets for this problem and show a consistent improvement in time series prediction results.

[1]  Partha Talukdar,et al.  HyTE: Hyperplane-based Temporally aware Knowledge Graph Embedding , 2018, EMNLP.

[2]  Daniel Neil,et al.  Interpretable Graph Convolutional Neural Networks for Inference on Noisy Knowledge Graphs , 2018, ArXiv.

[3]  Pedro A. Szekely,et al.  Recurrent Event Network for Reasoning over Temporal Knowledge Graphs , 2019, ArXiv.

[4]  Mathias Niepert,et al.  Learning Sequence Encoders for Temporal Knowledge Graph Completion , 2018, EMNLP.

[5]  Yan Liu,et al.  DynGEM: Deep Embedding Method for Dynamic Graphs , 2018, ArXiv.

[6]  Cyrus Shahabi,et al.  Graph Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ArXiv.

[7]  Jure Leskovec,et al.  GraphRNN: A Deep Generative Model for Graphs , 2018, ICML 2018.

[8]  Jianfeng Gao,et al.  Embedding Entities and Relations for Learning and Inference in Knowledge Bases , 2014, ICLR.

[9]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[10]  Le Song,et al.  Know-Evolve: Deep Reasoning in Temporal Knowledge Graphs , 2017, ArXiv.

[11]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[12]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[13]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[14]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[15]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[16]  Xing Xie,et al.  Discovering spatio-temporal causal interactions in traffic data streams , 2011, KDD.

[17]  John Fox,et al.  The Knowledge Engineering Review , 1984, The Knowledge Engineering Review.

[18]  Jie Chen,et al.  EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs , 2020, AAAI.

[19]  Paolo Frasconi,et al.  Short-Term Traffic Flow Forecasting: An Experimental Comparison of Time-Series Analysis and Supervised Learning , 2013, IEEE Transactions on Intelligent Transportation Systems.

[20]  R. Zemel,et al.  Neural Relational Inference for Interacting Systems , 2018, ICML.

[21]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[22]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.