A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution

Event coreference resolution is an important research problem with many applications. Despite the recent remarkable success of pre-trained language models, we argue that it is still highly beneficial to utilize symbolic features for the task. However, as the input for coreference resolution typically comes from upstream components in the information extraction pipeline, the automatically extracted symbolic features can be noisy and contain errors. Also, depending on the specific context, some features can be more informative than others. Motivated by these observations, we propose a novel context-dependent gated module to adaptively control the information flows from the input symbolic features. Combined with a simple noisy training method, our best models achieve state-of-the-art results on two datasets: ACE 2005 and KBP 2016.

[1]  Lysandre Debut,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[2]  Heng Ji,et al.  Graph-based Event Coreference Resolution , 2009, Graph-based Methods for Natural Language Processing.

[3]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[4]  Chen Chen,et al.  Joint Inference over a Lightly Supervised Information Extraction Pipeline: Towards Event Coreference Resolution for Resource-Scarce Languages , 2016, AAAI.

[5]  Jing Lu,et al.  Joint Learning for Event Coreference Resolution , 2017, ACL.

[6]  Ruihong Huang,et al.  Identifying the Most Dominant Event in a News Article by Mining Event Coreference Relations , 2018, NAACL-HLT.

[7]  Jing Lu,et al.  Event Coreference Resolution: A Survey of Two Decades of Research , 2018, IJCAI.

[8]  Hao Wu,et al.  UI CCG TAC-KBP2017 Submissions: Entity Discovery and Linking, and Event Nugget Detection and Co-reference , 2017, TAC.

[9]  Jing Lu,et al.  Event Coreference Resolution with Multi-Pass Sieves , 2016, LREC.

[10]  Omer Levy,et al.  SpanBERT: Improving Pre-training by Representing and Predicting Spans , 2019, TACL.

[11]  Ralph Grishman,et al.  New York University 2016 System for KBP Event Nugget: A Deep Learning Approach , 2016, TAC.

[12]  Heng Ji,et al.  Reliability-aware Dynamic Feature Composition for Name Tagging , 2019, ACL.

[13]  Jialong Tang,et al.  End-to-End Neural Event Coreference Resolution , 2020, Artif. Intell..

[14]  Luke S. Zettlemoyer,et al.  End-to-end Neural Coreference Resolution , 2017, EMNLP.

[15]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[16]  Dan Roth,et al.  Illinois CCG TAC 2015 Event Nugget, Entity Discovery and Linking, and Slot Filler Validation Systems , 2015, TAC.

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[19]  Jun Zhao,et al.  Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks , 2015, ACL.

[20]  K. Jarrod Millman,et al.  Array programming with NumPy , 2020, Nat..

[21]  Heng Ji,et al.  Cross-document Event Coreference Resolution based on Cross-media Features , 2015, EMNLP.

[22]  Chen Chen,et al.  SinoCoreferencer: An End-to-End Chinese Event Coreference Resolver , 2014, LREC.

[23]  M. Felisa Verdejo,et al.  Events are Not Simple: Identity, Non-Identity, and Quasi-Identity , 2013, EVENTS@NAACL-HLT.

[24]  Jing Lu,et al.  Improving Event Coreference Resolution by Learning Argument Compatibility from Unlabeled Data , 2019, NAACL.

[25]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[26]  Michele Banko,et al.  Event-Centric Summary Generation , 2004 .

[27]  Heng Ji,et al.  Knowledge Base Population: Successful Approaches and Challenges , 2011, ACL.

[28]  Heng Ji,et al.  A Pairwise Event Coreference Model, Feature Impact and Evaluation for Event Coreference Resolution , 2009 .

[29]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[30]  Dan Roth,et al.  Paired Representation Learning for Event and Entity Coreference , 2020, ArXiv.

[31]  Eduard Hovy,et al.  Identity, non-identity, and near-identity: Addressing the complexity of coreference , 2011 .

[32]  Ruihong Huang,et al.  Event Coreference Resolution by Iteratively Unfolding Inter-dependencies among Events , 2017, EMNLP.

[33]  Teruko Mitamura,et al.  Overview of TAC KBP 2015 Event Nugget Track , 2015, TAC.

[34]  Quan Hung Tran,et al.  A Gated Self-attention Memory Network for Answer Selection , 2019, EMNLP.

[35]  Ying Lin,et al.  A Joint Neural Model for Information Extraction with Global Features , 2020, ACL.

[36]  Jing Lu,et al.  UTD’s Event Nugget Detection and Coreference System at KBP 2017 , 2017, TAC.

[37]  Jing Lu,et al.  Learning Antecedent Structures for Event Coreference Resolution , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[38]  Dan Roth,et al.  Event Detection and Co-reference with Minimal Supervision , 2016, EMNLP.