Dispatched attention with multi-task learning for nested mention recognition

Abstract Entity mentions usually contain other mention in the task of named entity recognition (NER). Nested entities pose challenge to the task of NER. Existing methods fail to sufficiently capture the boundaries information between nested entities, which limits the performance of the task. In this paper, we propose a dispatched attention neural model with multi-task learning for the task. In particular, given an input sentence, a bi-directional Long Short Term Memory (BiLSTM) encodes it as common contextualized hidden representation. Then position and syntax information are leveraged into attention network for capturing mention span features. The attention representation of each task is dispatched to subsequent task to exchange boundaries information for nested mentions. Finally, Conditional Random Fields (CRFs) are used to extract nested mentions in an inside-out order for each task. Results on ACE2005 and GENIA datasets show that the proposed model outperforms state-of-the-art systems, showing its effectiveness in detecting nested mentions.

[1]  Deyu Zhou,et al.  Position-aware deep multi-task learning for drug-drug interaction extraction , 2018, Artif. Intell. Medicine.

[2]  Han Ren,et al.  Context-augmented convolutional neural networks for twitter sarcasm detection , 2018, Neurocomputing.

[3]  Christopher D. Manning,et al.  Nested Named Entity Recognition , 2009, EMNLP.

[4]  Hui Jiang,et al.  A Local Detection Approach for Named Entity Recognition and Mention Detection , 2017, ACL.

[5]  Dan Roth,et al.  Joint Mention Extraction and Classification with Mention Hypergraphs , 2015, EMNLP.

[6]  Jun'ichi Tsujii,et al.  GENIA corpus - a semantically annotated corpus for bio-textmining , 2003, ISMB.

[7]  Ting Wang,et al.  Using semantic similarity to reduce wrong labels in distant supervision for relation extraction , 2018, Inf. Process. Manag..

[8]  Yafeng Ren,et al.  Implicit Objective Network for Emotion Detection , 2019, NLPCC.

[9]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[10]  Hui Nie.  Person-specific named entity recognition using SVM with rich feature sets , 2012 .

[11]  Xiaohui Liang,et al.  A hybrid neural network model for predicting kidney disease in hypertension patients based on electronic health records , 2019, BMC Medical Informatics and Decision Making.

[12]  Laizhong Cui,et al.  Combine HowNet lexicon to train phrase recursive autoencoder for sentence-level sentiment analysis , 2017, Neurocomputing.

[13]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[14]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[15]  Han Ren,et al.  Neural Networks for Bacterial Named Entity Recognition , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[16]  Beatrice Alex,et al.  Recognising Nested Named Entities in Biomedical Text , 2007, BioNLP@ACL.

[17]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[18]  Donghong Ji,et al.  Long short-term memory RNN for biomedical named entity recognition , 2017, BMC Bioinformatics.

[19]  Xiaolin Li,et al.  GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text , 2017, Bioinform..

[20]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[21]  Dan Roth,et al.  The Importance of Syntactic Parsing and Inference in Semantic Role Labeling , 2008, CL.

[22]  Jian Wang,et al.  Biomedical event extraction based on GRU integrating attention mechanism , 2018, BMC Bioinformatics.

[23]  Sophia Ananiadou,et al.  A Neural Layered Model for Nested Named Entity Recognition , 2018, NAACL.

[24]  Luke S. Zettlemoyer,et al.  Deep Semantic Role Labeling: What Works and What’s Next , 2017, ACL.

[25]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[26]  Claire Cardie,et al.  Nested Named Entity Recognition Revisited , 2018, NAACL.

[27]  Andrew McCallum,et al.  Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets , 2018, EMNLP.

[28]  Chien-Chang Chen,et al.  Detecting spamming reviews using long short-term memory recurrent neural network framework , 2018, ICEEG '18.

[29]  Ikuya Yamada,et al.  Linkify: Enhancing Text Reading Experience by Detecting and Linking Helpful Entities to Users , 2018, IEEE Intelligent Systems.

[30]  Danqi Chen,et al.  Position-aware Attention and Supervised Data Improve Slot Filling , 2017, EMNLP.

[31]  Shao-Wu Zhang,et al.  NPBSS: a new PacBio sequencing simulator for generating the continuous long reads with an empirical model , 2018, BMC Bioinformatics.