Chinese Short Text Entity Disambiguation Based on the Dual-Channel Hybrid Network

Entity disambiguation refers to the accurate inference of the real mention of an entity with the same name according to the context. Most existing studies focused on long texts, for short texts, the performance has been unsatisfactory due to sparsity. In this paper, we treat the entity disambiguation task as a classification problem. we propose a novel neural network-based capsule network and convolutional neural network for entity disambiguation, leveraging full semantic information of short text data. In particular, a self-attention mechanism is utilized to further filter the semantic information extracted from the capsule network. On the other hand, a convolutional neural network with combined pooling is established to capture semantics from another channel. In the end, the semantic features obtained by the above models are combined through a fully connected layer to complete the task of entity disambiguation. The experimental results on the CCKS 2019 entity linking dataset showed that the dual-channel hybrid network proposed in this paper achieved an F1-score of 88.04%, which is superior to that of the existing mainstream deep learning model, thereby verifying the effectiveness of the model.

[1]  Geoffrey E. Hinton,et al.  Transforming Autoencoders , 2011 .

[2]  Breck Baldwin,et al.  Entity-Based Cross-Document Coreferencing Using the Vector Space Model , 1998, COLING.

[3]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[4]  Dan Roth,et al.  Entity Linking via Joint Encoding of Types, Descriptions, and Context , 2017, EMNLP.

[5]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[6]  Samuel Broscheit,et al.  Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking , 2019, CoNLL.

[7]  Qingshan Jiang,et al.  Feature selection via maximizing global information gain for text classification , 2013, Knowl. Based Syst..

[8]  Philip S. Yu,et al.  BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis , 2019, NAACL.

[9]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[10]  Zhenjin Dai,et al.  Named Entity Recognition Using BERT BiLSTM CRF for Chinese Electronic Health Records , 2019, 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[11]  Leonardo Neves,et al.  Multimodal Named Entity Disambiguation for Noisy Social Media Posts , 2018, ACL.

[12]  Houfeng Wang,et al.  Learning Entity Representation for Entity Disambiguation , 2013, ACL.

[13]  Jimmy J. Lin,et al.  Simple BERT Models for Relation Extraction and Semantic Role Labeling , 2019, ArXiv.

[14]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[15]  Romaric Besançon,et al.  Apprendre des représentations jointes de mots et d’entités pour la désambiguïsation d’entités (Combining Word and Entity Embeddings for Entity Linking) , 2017, JEPTALNRECITAL.

[16]  Dalwinder Singh,et al.  Hybridization of feature selection and feature weighting for high dimensional data , 2018, Applied Intelligence.

[17]  Min Yang,et al.  Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications , 2019, ACL.

[18]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[19]  Gerhard Weikum,et al.  diaNED: Time-Aware Named Entity Disambiguation for Diachronic Corpora , 2018, ACL.

[20]  Antonio Maria Rinaldi,et al.  A content-based approach for document representation and retrieval , 2008, DocEng '08.

[21]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[22]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[23]  Min Yang,et al.  Investigating Capsule Networks with Dynamic Routing for Text Classification , 2018, EMNLP.

[24]  Ebrahim Bagheri,et al.  Open Information Extraction , 2016, Encycl. Semantic Comput. Robotic Intell..

[25]  Xiaoyan Zhu,et al.  Sentiment Analysis by Capsules , 2018, WWW.

[26]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[27]  Laith Mohammad Abualigah,et al.  Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering , 2017, The Journal of Supercomputing.

[28]  Jian Xu,et al.  High Performance Clustering for Web Person Name Disambiguation Using Topic Capturing , 2011 .

[29]  Rajeev Rastogi,et al.  Entity disambiguation with hierarchical topic models , 2011, KDD.

[30]  Alexander M. Rush,et al.  Structured Attention Networks , 2017, ICLR.

[31]  Yi Tay,et al.  Pair-Linking for Collective Entity Disambiguation: Two Could Be Better Than All , 2018, IEEE Transactions on Knowledge and Data Engineering.

[32]  Jin Wang,et al.  Combining Knowledge with Deep Convolutional Neural Networks for Short Text Classification , 2017, IJCAI.

[33]  Wei Zhao,et al.  Investigating the transferring capability of capsule networks for text classification , 2019, Neural Networks.

[34]  Weidong Xiao,et al.  Entity Linking via Symmetrical Attention-Based Neural Network and Entity Structural Features , 2019, Symmetry.

[35]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[36]  Qiang Zhou,et al.  Leveraging Conceptualization for Short-Text Embedding , 2018, IEEE Transactions on Knowledge and Data Engineering.

[37]  Danushka Bollegala,et al.  Disambiguating Personal Names on the Web Using Automatically Extracted Key Phrases , 2006, ECAI.

[38]  Wei Zhang,et al.  Attention-Based Capsule Networks with Dynamic Routing for Relation Extraction , 2018, EMNLP.