Bilinear joint learning of word and entity embeddings for Entity Linking

Abstract Entity Linking (EL) is the task of resolving mentions to referential entities in a knowledge base, which facilitates applications such as information retrieval, question answering, and knowledge base population. In this paper, we propose a novel embedding method specifically designed for EL. The proposed model jointly learns word and entity embeddings which are located in different distributed spaces, and a bilinear model is introduced to simulate the interaction between words and entities. We treat EL as a ranking problem, and utilize a pairwise learning-to-rank framework with features constructed with learned embeddings as well as conventional EL features. Experimental results show the proposed model produces effective embeddings which improve the performance of our EL algorithm. Our method yields the state-of-the-art performances on two benchmark datasets CoNLL and TAC-KBP 2010.

[1]  Jennifer Widom,et al.  Scaling personalized web search , 2003, WWW '03.

[2]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[3]  Robert J. Gaizauskas,et al.  Graph Ranking for Collective Named Entity Disambiguation , 2014, ACL.

[4]  Tingting Mu,et al.  Translating on pairwise entity space for knowledge graph embedding , 2017, Neurocomputing.

[5]  Aapo Hyvärinen,et al.  Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics , 2012, J. Mach. Learn. Res..

[6]  Wei Shen,et al.  LINDEN: linking named entities with knowledge base via semantic knowledge , 2012, WWW.

[7]  Xiaolong Wang,et al.  Modeling Mention, Context and Entity with Neural Networks for Entity Disambiguation , 2015, IJCAI.

[8]  Fernando Pereira,et al.  Collective Entity Resolution with Multi-Focal Attention , 2016, ACL.

[9]  Yifan He,et al.  Personalized Page Rank for Named Entity Disambiguation , 2015, NAACL.

[10]  Dan Klein,et al.  Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks , 2016, NAACL.

[11]  Tommy W. S. Chow,et al.  Organizing Books and Authors by Multilayer SOM , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[13]  Ian H. Witten,et al.  Learning to link with wikipedia , 2008, CIKM '08.

[14]  Zhiyuan Liu,et al.  Learning Entity and Relation Embeddings for Knowledge Graph Completion , 2015, AAAI.

[15]  Gerhard Weikum,et al.  Robust Disambiguation of Named Entities in Text , 2011, EMNLP.

[16]  Oliver Ferschke,et al.  Wikipedia Revision Toolkit: Efficiently Accessing Wikipedia’s Edit History , 2011, ACL.

[17]  Eric P. Xing,et al.  Entity Hierarchy Embedding , 2015, ACL.

[18]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[19]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[20]  Heng Ji,et al.  Overview of the TAC 2010 Knowledge Base Population Track , 2010 .

[21]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[22]  Larry P. Heck,et al.  Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation , 2015, ArXiv.

[23]  Hiroyuki Shindo,et al.  Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation , 2016, CoNLL.

[24]  Zellig S. Harris,et al.  Distributional Structure , 1954 .