MZET: Memory Augmented Zero-Shot Fine-grained Named Entity Typing

Named entity typing (NET) is a classification task of assigning an entity mention in the context with given semantic types. However, with the growing size and granularity of the entity types, rare researches in previous concern with newly emerged entity types. In this paper, we propose MZET, a novel memory augmented FNET (Fine-grained NET) model, to tackle the unseen types in a zero-shot manner. MZET incorporates character-level, word-level, and contextural-level information to learn the entity mention representation. Besides, MZET considers the semantic meaning and the hierarchical structure into the entity type representation. Finally, through the memory component which models the relationship between the entity mention and the entity type, MZET transfer the knowledge from seen entity types to the zero-shot ones. Extensive experiments on three public datasets show prominent performance obtained by MZET, which surpasses the state-of-the-art FNET neural network models with up to 7\% gain in Micro-F1 and Macro-F1 score.

[1]  Ashish Anand,et al.  Fine-Grained Entity Type Classification by Jointly Learning Representations and Label Embeddings , 2017, EACL.

[2]  Erik Cambria,et al.  Label Embedding for Zero-shot Fine-grained Named Entity Typing , 2016, COLING.

[3]  Erik F. Tjong Kim Sang,et al.  Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition , 2003, CoNLL.

[4]  Praveen Paritosh,et al.  Freebase: a collaboratively created graph database for structuring human knowledge , 2008, SIGMOD Conference.

[5]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[6]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[7]  Dan Roth,et al.  Zero-Shot Open Entity Typing as Type-Compatible Grounding , 2019, EMNLP.

[8]  Daniel S. Weld,et al.  Fine-Grained Entity Recognition , 2012, AAAI.

[9]  Heng Ji,et al.  Building a Fine-Grained Entity Typing System Overnight for a New X (X = Language, Domain, Genre) , 2016, ArXiv.

[10]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[11]  Xiaoli Z. Fern,et al.  Description-Based Zero-shot Fine-Grained Entity Typing , 2019, NAACL.

[12]  Shafiq Joty,et al.  Zero-Resource Cross-Lingual Named Entity Recognition , 2020, AAAI.

[13]  Doug Downey,et al.  OTyper: A Neural Architecture for Open Named Entity Typing , 2018, AAAI.

[14]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[15]  Gerhard Weikum,et al.  Fine-grained Semantic Typing of Emerging Entities , 2013, ACL.

[16]  Heng Ji,et al.  AFET: Automatic Fine-Grained Entity Typing by Hierarchical Partial-Label Embedding , 2016, EMNLP.

[17]  Gerhard Weikum,et al.  FINET: Context-Aware Fine-Grained Named Entity Typing , 2015, EMNLP.

[18]  Eduard H. Hovy,et al.  Fine Grained Classification of Named Entities , 2002, COLING.

[19]  Frédéric Béchet,et al.  A Joint Named Entity Recognition and Entity Linking System , 2012 .

[20]  Mitchell P. Marcus,et al.  OntoNotes : A Large Training Corpus for Enhanced Processing , 2017 .

[21]  Philip S. Yu,et al.  Multi-grained Named Entity Recognition , 2019, ACL.

[22]  Wei Lu,et al.  Neural Adaptation Layers for Cross-domain Named Entity Recognition , 2018, EMNLP.

[23]  Philip S. Yu,et al.  Zero-shot User Intent Detection via Capsule Neural Networks , 2018, EMNLP.

[24]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[25]  Yang Liu,et al.  Exploring Fine-grained Entity Type Constraints for Distantly Supervised Relation Extraction , 2014, COLING.

[26]  Kentaro Inui,et al.  Neural Architectures for Fine-grained Entity Type Classification , 2016, EACL.

[27]  Sangdo Han,et al.  Answer ranking based on named entity types for question answering , 2017, IMCOM.

[28]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[29]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.