Understanding Event Predictions via Contextualized Multilevel Feature Learning

Deep learning models have been studied to forecast human events using vast volumes of data, yet they still cannot be trusted in certain applications such as healthcare and disaster assistance due to the lack of interpretability. Providing explanations for event predictions not only helps practitioners understand the underlying mechanism of prediction behavior but also enhances the robustness of event analysis. Improving the transparency of event prediction models is challenging given the following factors: (i) multilevel features exist in event data which creates a challenge to cross-utilize different levels of data; (ii) features across different levels and time steps are heterogeneous and dependent; and (iii) static model-level interpretations cannot be easily adapted to event forecasting given the dynamic and temporal characteristics of the data. Recent interpretation methods have proven their capabilities in tasks that deal with graph-structured or relational data. In this paper, we present a Contextualized Multilevel Feature learning framework, CMF, for interpretable temporal event prediction. It consists of a predictor for forecasting events of interest and an explanation module for interpreting model predictions. We design a new context-based feature fusion method to integrate multiple levels of heterogeneous features. We also introduce a temporal explanation module to determine sequences of text and subgraphs that have crucial roles in a prediction. We conduct extensive experiments on several real-world datasets of political and epidemic events. We demonstrate that the proposed method is competitive compared with the state-of-the-art models while possessing favorable interpretation capabilities.

[1]  Volker Tresp,et al.  Explainable Subgraph Reasoning for Forecasting on Temporal Knowledge Graphs , 2021, ICLR.

[2]  Huzefa Rangwala,et al.  Learning Dynamic Context Graphs for Predicting Social Events , 2019, KDD.

[3]  M. Shahriar Hossain,et al.  Storytelling in entity networks to support intelligence analysts , 2012, KDD.

[4]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[5]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[6]  Xia Hu,et al.  Techniques for interpretable machine learning , 2018, Commun. ACM.

[7]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[8]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[9]  Bo Zong,et al.  Parameterized Explainer for Graph Neural Network , 2020, NeurIPS.

[10]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[11]  Naren Ramakrishnan,et al.  STAPLE: Spatio-Temporal Precursor Learning for Event Forecasting , 2018, SDM.

[12]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[13]  Yanfang Ye,et al.  Incomplete Label Multi-Task Deep Learning for Spatio-Temporal Event Subtype Forecasting , 2019, AAAI.

[14]  Matthew S. Gerber,et al.  Predicting crime using Twitter and kernel density estimation , 2014, Decis. Support Syst..

[15]  Maosong Sun,et al.  ERNIE: Enhanced Language Representation with Informative Entities , 2019, ACL.

[16]  Jure Leskovec,et al.  GNNExplainer: Generating Explanations for Graph Neural Networks , 2019, NeurIPS.

[17]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[18]  Le Song,et al.  Learning to Explain: An Information-Theoretic Perspective on Model Interpretation , 2018, ICML.

[19]  Isabell M. Welpe,et al.  Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment , 2010, ICWSM.

[20]  Naren Ramakrishnan,et al.  A Multiple Instance Learning Framework for Identifying Key Sentences and Detecting Events , 2016, CIKM.

[21]  Liang Zhao,et al.  Multi-resolution Spatial Event Forecasting in Social Media , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[22]  Alberto Maria Segre,et al.  The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic , 2011, PloS one.

[23]  Xiaohui Xie,et al.  Dynamically Pruned Message Passing Networks for Large-Scale Knowledge Graph Reasoning , 2020, ICLR.

[24]  Xiaofeng Wang,et al.  Automatic Crime Prediction Using Events Extracted from Twitter Posts , 2012, SBP.

[25]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[26]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[27]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[28]  Matteo Pagliardini,et al.  Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features , 2017, NAACL.

[29]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[30]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[31]  Aravind Srinivasan,et al.  'Beating the news' with EMBERS: forecasting civil unrest using open source indicators , 2014, KDD.

[32]  Ramón Fernández Astudillo,et al.  From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification , 2016, ICML.

[33]  Huzefa Rangwala,et al.  Dynamic Knowledge Graph based Multi-Event Forecasting , 2020, KDD.

[34]  Johan Bollen,et al.  Twitter mood predicts the stock market , 2010, J. Comput. Sci..

[35]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[36]  Naren Ramakrishnan,et al.  Modeling Precursors for Event Forecasting via Nested Multi-Instance Learning , 2016, KDD.

[37]  Bernd Bischl,et al.  iml: An R package for Interpretable Machine Learning , 2018, J. Open Source Softw..

[38]  Shuiwang Ji,et al.  XGNN: Towards Model-Level Explanations of Graph Neural Networks , 2020, KDD.

[39]  Pedro A. Szekely,et al.  Recurrent Event Network for Reasoning over Temporal Knowledge Graphs , 2019, ArXiv.

[40]  Jieping Ye,et al.  Multi-Task Learning for Spatio-Temporal Event Forecasting , 2015, KDD.