Globally normalized neural model for joint entity and event extraction

Abstract Extracting events from texts using neural networks has gained increasing research focus in recent years. However, existing methods prepare candidate arguments in a separate classifier suffering from the error propagation problem and fail to model correlations between entity mentions and event structures. To improve the performance of both entity recognition and event extraction, we propose a transition-based joint neural model for the tasks by converting graph structures to a set of transition actions. In particular, we design ten types of novel actions and introduce a global normalization strategy to alleviate the label bias issue. We conduct experiments based on the widely used Automatic Content Extraction (ACE) corpora and the results show that our model achieves 88.7% F1-score on entities and 75.3% F1-score on event triggers, outperforming the baseline neural networks by a large margin. Further in-depth analysis shows the effectiveness of our model in capturing structural dependencies in long sentences. The proposed model can be used for facilitating a range of downstream tasks.

[1]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[2]  Heng Ji,et al.  Refining Event Extraction through Cross-Document Inference , 2008, ACL.

[3]  Zhifang Sui,et al.  Jointly Extracting Event Triggers and Arguments by Dependency-Bridge RNN and Tensor-Based Argument Interaction , 2018, AAAI.

[4]  Yafeng Ren,et al.  A tree-based neural network model for biomedical event trigger detection , 2020, Inf. Sci..

[5]  Weiguo Fan,et al.  Adverse drug event detection and extraction from open data: A deep learning approach , 2020, Inf. Process. Manag..

[6]  Danqi Chen,et al.  Position-aware Attention and Supervised Data Improve Slot Filling , 2017, EMNLP.

[7]  Qing Liu,et al.  Graph Convolution Over Multiple Latent Context-Aware Graph Structures for Event Detection , 2020, IEEE Access.

[8]  Thien Huu Nguyen,et al.  One for All: Neural Joint Modeling of Entities and Events , 2018, AAAI.

[9]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[10]  Ralph Grishman,et al.  Using Document Level Cross-Event Inference to Improve Event Extraction , 2010, ACL.

[11]  Heng Ji,et al.  Constructing Information Networks Using One Single Model , 2014, EMNLP.

[12]  Tom M. Mitchell,et al.  Joint Extraction of Events and Entities within a Document Context , 2016, NAACL.

[13]  Bang Wang,et al.  A Survey of Event Extraction From Text , 2019, IEEE Access.

[14]  Yongliang Wu,et al.  A novel community answer matching approach based on phrase fusion heterogeneous information network , 2021, Inf. Process. Manag..

[15]  Marjorie Freedman,et al.  Contextualized Cross-Lingual Event Trigger Extraction with Minimal Resources , 2019, CoNLL.

[16]  Stephen Clark,et al.  Syntactic Processing Using the Generalized Perceptron and Beam Search , 2011, CL.

[17]  Dong-Hong Ji,et al.  Document-level event causality identification via graph inference mechanism , 2021, Inf. Sci..

[18]  Joakim Nivre,et al.  Algorithms for Deterministic Incremental Dependency Parsing , 2008, CL.

[19]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[20]  Qi He,et al.  Syntax grounded graph convolutional network for joint entity and event extraction , 2021, Neurocomputing.

[21]  Guodong Zhou,et al.  Tree kernel-based semantic relation extraction with rich syntactic and semantic information , 2010, Inf. Sci..

[22]  Jakub Piskorski,et al.  Shallow and deep learning for event relatedness classification , 2020, Inf. Process. Manag..

[23]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[24]  Donghong Ji,et al.  Extracting Entities and Events as a Single Task Using a Transition-Based Neural Model , 2019, IJCAI.

[25]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[26]  Jun Zhao,et al.  Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks , 2015, ACL.

[27]  Yue Zhang,et al.  Joint Word Segmentation, POS-Tagging and Syntactic Chunking , 2016, AAAI.

[28]  Ying Lin,et al.  A Joint Neural Model for Information Extraction with Global Features , 2020, ACL.

[29]  Ralph Grishman,et al.  Joint Event Extraction via Recurrent Neural Networks , 2016, NAACL.

[30]  Yue Zhang,et al.  A Transition-based Model for Joint Segmentation, POS-tagging and Normalization , 2015, EMNLP.

[31]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[32]  Hannaneh Hajishirzi,et al.  Entity, Relation, and Event Extraction with Contextualized Span Representations , 2019, EMNLP.

[33]  Amir Pouran Ben Veyseh,et al.  Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction , 2020, FINDINGS.

[34]  Thien Huu Nguyen,et al.  Event Detection: Gate Diversity and Syntactic Importance Scores for Graph Convolution Neural Networks , 2020, EMNLP.

[35]  Dongsheng Li,et al.  Exploring Pre-trained Language Models for Event Extraction and Generation , 2019, ACL.

[36]  Jian Liu,et al.  Event Detection via Gated Multilingual Attention Mechanism , 2018, AAAI.

[37]  Jun Zhao,et al.  Collective Event Detection via a Hierarchical and Bias Tagging Networks with Gated Multi-level Attention Mechanisms , 2018, EMNLP.

[38]  Yu Hong,et al.  Interactive learning for joint event and relation extraction , 2020, Int. J. Mach. Learn. Cybern..

[39]  Jing Liu,et al.  RBPB: Regularization-Based Pattern Balancing Method for Event Extraction , 2016, ACL.

[40]  Feras Al-Obeidat,et al.  Extracting temporal and causal relations based on event networks , 2020, Inf. Process. Manag..

[41]  Michael Strube,et al.  Incremental Global Event Extraction , 2016, COLING.

[42]  Yang Wang,et al.  Empower event detection with bi-directional neural language model , 2019, Knowl. Based Syst..

[43]  Heng Ji,et al.  Joint Event Extraction via Structured Prediction with Global Features , 2013, ACL.

[44]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[45]  Heng Ji,et al.  Joint Entity and Event Extraction with Generative Adversarial Imitation Learning , 2019, Data Intelligence.

[46]  Xiao Liu,et al.  Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation , 2018, EMNLP.

[47]  Mihai Surdeanu,et al.  Event Extraction as Dependency Parsing , 2011, ACL.

[48]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[49]  Wanxiang Che,et al.  A Neural Transition-Based Approach for Semantic Dependency Graph Parsing , 2018, AAAI.