Improving Neural Fine-Grained Entity Typing With Knowledge Attention

Fine-grained entity typing aims to identify the semantic type of an entity in a particular plain text. It is an important task which can be helpful for a lot of natural language processing (NLP) applications. Most existing methods typically extract features separately from the entity mention and context words for type classification. These methods inevitably fail to model complex correlations between entity mentions and context words. They also neglect rich background information about these entities in knowledge bases (KBs). To address these issues, we take information from KBs into consideration to bridge entity mentions and their context together, and thereby propose Knowledge-Attention Neural Fine-Grained Entity Typing. Experimental results and case studies on real-world datasets demonstrate that our model significantly outperforms other state-of-the-art methods, revealing the effectiveness of incorporating KB information for entity typing. Code and data for this paper can be found at https://github.com/thunlp/KNET.

[1]  Yoram Singer,et al.  Unsupervised Models for Named Entity Classification , 1999, EMNLP.

[2]  Erik F. Tjong Kim Sang,et al.  Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition , 2003, CoNLL.

[3]  Zornitsa Kozareva.  Bootstrapping Named Entity Recognition with Automatically Generated Gazetteer Lists , 2006, EACL.

[4]  ChengXiang Zhai,et al.  Exploiting Domain Structure for Named Entity Recognition , 2006, NAACL.

[5]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[6]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[7]  Estevam R. Hruschka,et al.  Coupled semi-supervised learning for information extraction , 2010, WSDM '10.

[8]  Jason Weston,et al.  Learning Structured Embeddings of Knowledge Bases , 2011, AAAI.

[9]  Daniel S. Weld,et al.  Fine-Grained Entity Recognition , 2012, AAAI.

[10]  Gerhard Weikum,et al.  HYENA: Hierarchical Type Classification for Entity Names , 2012, COLING.

[11]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[12]  Gerhard Weikum,et al.  Robust question answering over the web of linked data , 2013, CIKM.

[13]  Zhen Wang,et al.  Knowledge Graph Embedding by Translating on Hyperplanes , 2014, AAAI.

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Nevena Lazic,et al.  Context-Dependent Fine-Grained Entity Type Tagging , 2014, ArXiv.

[16]  Jason Weston,et al.  Open Question Answering with Weakly Supervised Embedding Models , 2014, ECML/PKDD.

[17]  Yang Liu,et al.  Exploring Fine-grained Entity Type Constraints for Distantly Supervised Relation Extraction , 2014, COLING.

[18]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[19]  Jun Zhao,et al.  Learning to Represent Knowledge Graphs with Gaussian Embedding , 2015, CIKM.

[20]  Jun Zhao,et al.  Knowledge Graph Embedding via Dynamic Mapping Matrix , 2015, ACL.

[21]  Gerhard Weikum,et al.  FINET: Context-Aware Fine-Grained Named Entity Typing , 2015, EMNLP.

[22]  Hinrich Schütze,et al.  Corpus-level Fine-grained Entity Typing Using Contextual Information , 2015, EMNLP.

[23]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[24]  Hong Sun,et al.  A Hybrid Neural Model for Type Classification of Entity Mentions , 2015, IJCAI.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[27]  Nevena Lazic,et al.  Embedding Methods for Fine Grained Entity Type Classification , 2015, ACL.

[28]  Heng Ji,et al.  Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding , 2016, KDD.

[29]  Kentaro Inui,et al.  An Attentive Neural Architecture for Fine-grained Entity Type Classification , 2016, AKBC@NAACL-HLT.

[30]  Amal Zouaq,et al.  Collective Disambiguation and Semantic Annotation for Entity Linking and Typing , 2016, SemWebEval@ESWC.

[31]  Lorenzo Rosasco,et al.  Holographic Embeddings of Knowledge Graphs , 2015, AAAI.

[32]  Jun Zhao,et al.  Knowledge Graph Completion with Adaptive Sparse Transfer Matrix , 2016, AAAI.

[33]  Heng Ji,et al.  AFET: Automatic Fine-Grained Entity Typing by Hierarchical Partial-Label Embedding , 2016, EMNLP.

[34]  Kentaro Inui,et al.  Neural Architectures for Fine-grained Entity Type Classification , 2016, EACL.

[35]  Heike Adel,et al.  Noise Mitigation for Neural Entity Typing and Relation Extraction , 2016, EACL.

[36]  Hinrich Schütze,et al.  Multi-level Representations for Fine-Grained Typing of Knowledge Base Entities , 2017, EACL.