Combined Self-Attention Mechanism for Chinese Named Entity Recognition in Military

Military named entity recognition (MNER) is one of the key technologies in military information extraction. Traditional methods for the MNER task rely on cumbersome feature engineering and specialized domain knowledge. In order to solve this problem, we propose a method employing a bidirectional long short-term memory (BiLSTM) neural network with a self-attention mechanism to identify the military entities automatically. We obtain distributed vector representations of the military corpus by unsupervised learning and the BiLSTM model combined with the self-attention mechanism is adopted to capture contextual information fully carried by the character vector sequence. The experimental results show that the self-attention mechanism can improve effectively the performance of MNER task. The F-score of the military documents and network military texts identification was 90.15% and 89.34%, respectively, which was better than other models.

[1]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[2]  Chandra Bhagavatula,et al.  Semi-supervised sequence tagging with bidirectional language models , 2017, ACL.

[3]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[4]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[5]  Eric Nichols,et al.  Named Entity Recognition with Bidirectional LSTM-CNNs , 2015, TACL.

[6]  Zdenek Zabokrtský,et al.  Czech Named Entity Corpus and SVM-based Recognizer , 2009, NEWS@IJCNLP.

[7]  Kamal Sarkar,et al.  An HMM Based Named Entity Recognition System for Indian Languages: The JU System at ICON 2013 , 2014, ArXiv.

[8]  Yidong Chen,et al.  Deep Semantic Role Labeling with Self-Attention , 2017, AAAI.

[9]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[10]  Heng Ji,et al.  Comparison of the Impact of Word Segmentation on Name Tagging for Chinese and Japanese , 2014, LREC.

[11]  Jürgen Schmidhuber,et al.  LSTM can Solve Hard Long Time Lag Problems , 1996, NIPS.

[12]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[13]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[14]  James R. Curran,et al.  Language Independent NER using a Maximum Entropy Tagger , 2003, CoNLL.

[15]  Tiejun Zhao,et al.  Chinese Named Entity Recognition with a Sequence Labeling Approach: Based on Characters, or Based on Words? , 2010, ICIC.

[16]  Ralph Grishman,et al.  Message Understanding Conference- 6: A Brief History , 1996, COLING.

[17]  Jinhong K. Guo,et al.  Extracting Meaningful Entities from Human-generated Tactical Reports , 2015, Complex Adaptive Systems.

[18]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[19]  Houfeng Wang,et al.  Chinese Named Entity Recognition and Word Segmentation Based on Character , 2008, IJCNLP.

[20]  Chengqing Zong,et al.  Approach to recognizing Uyghur names based on conditional random fields , 2013 .

[21]  Nanyun Peng,et al.  Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings , 2015, EMNLP.