Entity disambiguation with decomposable neural networks

Entity disambiguation is a fundamental task in natural language processing and computational linguistics. Given a query consisting of a mention (name string) and a background document, entity disambiguation aims at linking the mention to an entity from a reference knowledge base such as Wikipedia. A main challenge of this task is how to effectively represent the meaning of the mention and the entity, based on which the semantic relatedness between the mention and the entity could be conveniently measured. Towards this goal, we introduce computational models to effectively represent the mention and the entity in some vector space. We decompose the problem into subproblems and develop various neural network architectures, all of which are purely data‐driven and capable of learning continuous representations of the mention and the entity from data. To effectively train the neural network models, we explore a simple yet effective way that enables us to collect millions of training examples from Wikipedia without using any manual annotation. Empirical results on two benchmark datasets show that our approaches based on convolutional neural network and long short‐term memory consistently outperform top‐performed systems on both datasets. WIREs Data Mining Knowl Discov 2017, 7:e1215. doi: 10.1002/widm.1215

[1]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[2]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[3]  Heng Ji,et al.  Analysis and Enhancement of Wikification for Microblogs with Context Expansion , 2012, COLING.

[4]  Heng Ji,et al.  CUNY-UIUC-SRI TAC-KBP2011 Entity Linking System Description , 2011, TAC.

[5]  Rajeev Rastogi,et al.  Entity disambiguation with hierarchical topic models , 2011, KDD.

[6]  Eduard H. Hovy,et al.  Recursive Deep Models for Discourse Parsing , 2014, EMNLP.

[7]  Prithviraj Sen Collective context-aware topic models for entity disambiguation , 2012, WWW.

[8]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[9]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[10]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[11]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[12]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[13]  Sahin Albayrak,et al.  GerNED: A German Corpus for Named Entity Disambiguation , 2012, LREC.

[14]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[15]  G. Frege On Sense and Reference , 1948 .

[16]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[17]  Jun Zhao,et al.  Collective entity linking in web text: a graph-based method , 2011, SIGIR.

[18]  Yang Song,et al.  Efficient Collective Entity Linking with Stacking , 2013, EMNLP.

[19]  Houfeng Wang,et al.  Learning Entity Representation for Entity Disambiguation , 2013, ACL.

[20]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[21]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[22]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[23]  Andrew Y. Ng,et al.  Parsing with Compositional Vector Grammars , 2013, ACL.

[24]  Heng Ji,et al.  Knowledge Base Population: Successful Approaches and Challenges , 2011, ACL.

[25]  Iryna Gurevych,et al.  Extracting Lexical Semantic Knowledge from Wikipedia and Wiktionary , 2008, LREC.

[26]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[27]  G. Prasad LEARNING TO LINK ENTITIES WITH KNOWLEDGE BASE , 2016 .

[28]  Heng Ji,et al.  Collaborative Ranking: A Case Study on Entity Linking , 2011, EMNLP.

[29]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[30]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[31]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[32]  Mark Steedman,et al.  Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL) , 2015 .

[33]  Xianpei Han,et al.  A Generative Entity-Mention Model for Linking Entities with Knowledge Base , 2011, ACL.

[34]  Heng Ji,et al.  Overview of the TAC 2010 Knowledge Base Population Track , 2010 .

[35]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[36]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[37]  Doug Downey,et al.  Local and Global Algorithms for Disambiguation to Wikipedia , 2011, ACL.

[38]  Christian Bizer,et al.  DBpedia spotlight: shedding light on the web of documents , 2011, I-Semantics '11.

[39]  Joel Nothman,et al.  Evaluating Entity Linking with Wikipedia , 2013, Artif. Intell..