Probing and Fine-tuning Reading Comprehension Models for Few-shot Event Extraction

We study the problem of event extraction from text data, which requires both detecting target event types and their arguments. Typically, both the event detection and argument detection subtasks are formulated as supervised sequence labeling problems. We argue that the event extraction models so trained are inherently label-hungry, and can generalize poorly across domains and text genres.We propose a reading comprehension framework for event extraction.Specifically, we formulate event detection as a textual entailment prediction problem, and argument detection as a question answer-ing problem. By constructing proper query templates, our approach can effectively distill rich knowledge about tasks and label semantics from pretrained reading comprehension models. Moreover, our model can be fine-tuned with a small amount of data to boost its performance. Our experiment results show that our method performs strongly for zero-shot and few-shot event extraction, and it achieves state-of-the-art performance on the ACE 2005 benchmark when trained with full supervision.

[1]  Xu Han,et al.  Adversarial Training for Weakly Supervised Event Detection , 2019, NAACL.

[2]  Guodong Zhou,et al.  Self-regulation: Employing a Generative Adversarial Network to Improve Event Detection , 2018, ACL.

[3]  Jiwei Li,et al.  A Unified MRC Framework for Named Entity Recognition , 2019, ACL.

[4]  Ralph Grishman,et al.  Graph Convolutional Networks With Argument-Aware Pooling for Event Detection , 2018, AAAI.

[5]  Benjamin Van Durme,et al.  Reading the Manual: Event Extraction as Definition Comprehension , 2019, SPNLP.

[6]  Richard Socher,et al.  Unifying Question Answering, Text Classification, and Regression via Span Extraction , 2019 .

[7]  Ralph Grishman,et al.  Joint Event Extraction via Recurrent Neural Networks , 2016, NAACL.

[8]  Jun Zhao,et al.  A Probabilistic Soft Logic Based Approach to Exploiting Latent and Global Information in Event Classification , 2016, AAAI.

[9]  Lifu Huang,et al.  Zero-Shot Transfer Learning for Event Extraction , 2017, ACL.

[10]  Richard Socher,et al.  The Natural Language Decathlon: Multitask Learning as Question Answering , 2018, ArXiv.

[11]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[12]  W. Bruce Croft,et al.  Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2013 .

[13]  Sebastian Riedel,et al.  Language Models as Knowledge Bases? , 2019, EMNLP.

[14]  Yaojie Lu,et al.  Distilling Discrimination and Generalization Knowledge for Event Detection via Delta-Representation Learning , 2019, ACL.

[15]  Guilin Qi,et al.  Zero-Shot Slot Filling via Latent Question Representation and Reading Comprehension , 2019, PRICAI.

[16]  Haoran Yan,et al.  Event Detection with Multi-Order Graph Convolution and Aggregated Attention , 2019, EMNLP.

[17]  F. Massey.  The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[18]  Jun Zhao,et al.  Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks , 2015, ACL.

[19]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[20]  Zhifang Sui,et al.  Jointly Extracting Event Triggers and Arguments by Dependency-Bridge RNN and Tensor-Based Argument Interaction , 2018, AAAI.

[21]  Bin Ma,et al.  Using Cross-Entity Inference to Improve Event Extraction , 2011, ACL.

[22]  Samuel R. Bowman,et al.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , 2017, NAACL.

[23]  Heng Ji,et al.  Joint Event Extraction via Structured Prediction with Global Features , 2013, ACL.

[24]  Huajun Chen,et al.  Meta-Learning with Dynamic-Memory-Based Prototypical Network for Few-Shot Event Detection , 2020, WSDM.

[25]  Xiao Liu,et al.  Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation , 2018, EMNLP.

[26]  Ralph Grishman,et al.  Using Document Level Cross-Event Inference to Improve Event Extraction , 2010, ACL.

[27]  Jiwei Li,et al.  CorefQA: Coreference Resolution as Query-based Span Prediction , 2020, ACL.

[28]  Yang Li,et al.  Event Detection without Triggers , 2019, NAACL.

[29]  Andreas Vlachos,et al.  Zero-shot Relation Classification as Textual Entailment , 2018, FEVER@EMNLP.

[30]  J. L. Hodges,et al.  The significance probability of the smirnov two-sample test , 1958 .

[31]  Dan Roth,et al.  Event Detection and Co-reference with Minimal Supervision , 2016, EMNLP.

[32]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[33]  Ralph Grishman,et al.  Event Detection and Domain Adaptation with Convolutional Neural Networks , 2015, ACL.

[34]  Mohammad Raihanul Islam,et al.  Event Detection using Hierarchical Multi-Aspect Attention , 2019, WWW.

[35]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.

[36]  Steven Schockaert,et al.  Inducing Relational Knowledge from BERT , 2019, AAAI.

[37]  Rajarshi Das,et al.  Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension , 2018, ICLR.

[38]  Ruifang He,et al.  Exploiting Document Level Information to Improve Event Detection via Recurrent Neural Networks , 2017, IJCNLP.

[39]  Mike Schuster,et al.  Japanese and Korean voice search , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Xinya Du,et al.  Event Extraction by Answering (Almost) Natural Questions , 2020, EMNLP.

[41]  Yang Xiao,et al.  DCFEE: A Document-level Chinese Financial Event Extraction System based on Automatically Labeled Training Data , 2018, ACL.

[42]  Fei Wang,et al.  Coreference Resolution as Query-based Span Prediction , 2019, ArXiv.

[43]  Xiang Zhang,et al.  Automatically Labeled Data Generation for Large Scale Event Extraction , 2017, ACL.

[44]  Omer Levy,et al.  Zero-Shot Relation Extraction via Reading Comprehension , 2017, CoNLL.

[45]  Xiaoli Z. Fern,et al.  Event Detection with Neural Networks: A Rigorous Empirical Evaluation , 2018, EMNLP.

[46]  Colin Raffel,et al.  How Much Knowledge Can You Pack Into the Parameters of a Language Model? , 2020, EMNLP.

[47]  Dongsheng Li,et al.  Exploring Pre-trained Language Models for Event Extraction and Generation , 2019, ACL.

[48]  Jonathan Berant,et al.  Question Answering is a Format; When is it Useful? , 2019, ArXiv.