A Multi-domain Named Entity Recognition Method Based on Part-of-Speech Attention Mechanism

Named entity recognition is an important and basic work in text mining. To overcome the shortcomings of existing multi-domain named entity recognition methods, a multi-domain named entity recognition method based on the part-of-speech attention mechanism, called BiLSTM-ATTENTION-CRF, was proposed in this paper. The domain dictionary was constructed to represent multi-domain semantic information and the BiLSTM network was used to capture the grammatical and syntactic features, as well as multi-domain semantic features in context information. A part-of-speech attention mechanism was designed to obtain the contribution weight of part-of-speech for entity recognition. Finally, a group of experiments were performed on the multi-domain dataset to compare various fusion strategies of multi-level entity information. The experimental results show that BiLSTM-ATTENTION-CRF has a high precision and recall rate, and can effectively recognizes the multi-domain named entities.

[1]  Shaoli Liu,et al.  Cambricon: An Instruction Set Architecture for Neural Networks , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[2]  S Marrett,et al.  Local and global attention are mapped retinotopically in human occipital cortex. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[4]  Jürgen Schmidhuber,et al.  Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition , 2005, ICANN.

[5]  Jürgen Schmidhuber,et al.  Recurrent Highway Networks , 2016, ICML.

[6]  Hong Yu,et al.  Structured prediction models for RNN based sequence labeling in clinical text , 2016, EMNLP.

[7]  Eugénio C. Oliveira,et al.  A Bootstrapping Approach for Training a NER with Conditional Random Fields , 2011, EPIA.

[8]  Changjiang Zhou,et al.  Named entity recognition from Chinese adverse drug event reports with lexical feature based BiLSTM-CRF and tri-training , 2019, J. Biomed. Informatics.

[9]  Klaus Zechner,et al.  Using bidirectional lstm recurrent neural networks to learn high-level abstractions of sequential features for automated scoring of non-native spontaneous speech , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[10]  Wlodek Zadrozny,et al.  Mined semantic analysis: A new concept space model for semantic representation of textual data , 2015, 2017 IEEE International Conference on Big Data (Big Data).

[11]  Jianhui Chen,et al.  A narrow-domain entity recognition method based on domain relevance measurement and context information , 2017, WI.

[12]  Hai-Jun Huang,et al.  A multiclass, multicriteria logit-based traffic equilibrium assignment model under ATIS , 2007, Eur. J. Oper. Res..

[13]  Qian Liu,et al.  Improving Opinion Aspect Extraction Using Semantic Similarity and Aspect Associations , 2016, AAAI.

[14]  Paul S. Rosenbloom,et al.  Distributed Vector Representations of Words in the Sigma Cognitive Architecture , 2014, AGI.

[15]  Iryna Gurevych,et al.  Extracting Opinion Targets in a Single and Cross-Domain Setting with Conditional Random Fields , 2010, EMNLP.

[16]  Janna Lipenkova,et al.  A system for fine-grained aspect-based sentiment analysis of Chinese , 2015, ACL.

[17]  Xuanjing Huang,et al.  Multi-Timescale Long Short-Term Memory Neural Network for Modelling Sentences and Documents , 2015, EMNLP.

[18]  Wanxiang Che,et al.  Sentence Compression for Target-Polarity Word Collocation Extraction , 2014, COLING.

[19]  Marc Moens,et al.  Named Entity Recognition without Gazetteers , 1999, EACL.

[20]  Qiang Yang,et al.  Lifelong Machine Learning Systems: Beyond Learning Algorithms , 2013, AAAI Spring Symposium: Lifelong Machine Learning.

[21]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[22]  Tome Eftimov,et al.  A rule-based named-entity recognition method for knowledge extraction of evidence-based dietary recommendations , 2017, PloS one.

[23]  Shaofu Lin,et al.  Recognizing Small-Sample Biomedical Named Entity Based on Contextual Domain Relevance , 2019, 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[24]  Xuejie Zhang,et al.  An Attentive Neural Sequence Labeling Model for Adverse Drug Reactions Mentions Extraction , 2018, IEEE Access.

[25]  Richong Zhang,et al.  Prototypical Recurrent Unit , 2016, Neurocomputing.

[26]  Makoto Miwa,et al.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures , 2016, ACL.

[27]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[28]  Ariya Rastrow,et al.  Scalable Language Model Adaptation for Spoken Dialogue Systems , 2018, 2018 IEEE Spoken Language Technology Workshop (SLT).

[29]  Pierre Zweigenbaum,et al.  Medical Entity Recognition: A Comparaison of Semantic and Statistical Methods , 2011, BioNLP@ACL.

[30]  Sunita Sarawagi,et al.  Efficient Batch Top-k Search for Dictionary-based Entity Recognition , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[31]  Lucia Specia,et al.  Learning Structural Kernels for Natural Language Processing , 2015, TACL.

[32]  Deepti Chopra,et al.  Named Entity Recognition using Hidden Markov Model (HMM) , 2012 .

[33]  Fei Zhu,et al.  Named Entity Recognition from Biomedical Text Using SVM , 2011, 2011 5th International Conference on Bioinformatics and Biomedical Engineering.

[34]  Alexander M. Rush,et al.  LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.