Deep learning for named entity recognition on Chinese electronic medical records: Combining deep transfer learning with multitask bi-directional LSTM RNN

Specific entity terms such as disease, test, symptom, and genes in Electronic Medical Record (EMR) can be extracted by Named Entity Recognition (NER). However, limited resources of labeled EMR pose a great challenge for mining medical entity terms. In this study, a novel multitask bi-directional RNN model combined with deep transfer learning is proposed as a potential solution of transferring knowledge and data augmentation to enhance NER performance with limited data. The proposed model has been evaluated using micro average F-score, macro average F-score and accuracy. It is observed that the proposed model outperforms the baseline model in the case of discharge datasets. For instance, for the case of discharge summary, the micro average F-score is improved by 2.55% and the overall accuracy is improved by 7.53%. For the case of progress notes, the micro average F-score and the overall accuracy are improved by 1.63% and 5.63%, respectively.

[1]  Ruslan Salakhutdinov,et al.  Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks , 2016, ICLR.

[2]  Lijun Qian,et al.  Transfer bi-directional LSTM RNN for named entity recognition in Chinese electronic medical records , 2017, 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom).

[3]  Satoshi Sekine,et al.  A survey of named entity recognition and classification , 2007 .

[4]  Eric Nichols,et al.  Named Entity Recognition with Bidirectional LSTM-CNNs , 2015, TACL.

[5]  Yu Zhang,et al.  A Survey on Multi-Task Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[6]  Yiming Yang,et al.  A study of thresholding strategies for text categorization , 2001, SIGIR '01.

[7]  Lijun Qian,et al.  A multitask bi-directional RNN model for named entity recognition on Chinese electronic medical records , 2018, BMC Bioinformatics.

[8]  Chao Li,et al.  A Convolutional Neural Network Model for Online Medical Guidance , 2016, IEEE Access.

[9]  Gabriela Ferraro,et al.  Benchmarking Clinical Speech Recognition and Information Extraction: New Data, Methods, and Evaluations , 2015, JMIR medical informatics.

[10]  Parisa Rashidi,et al.  Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis , 2017, IEEE Journal of Biomedical and Health Informatics.

[11]  Lei Huang,et al.  A multiclass classification method based on deep learning for named entity recognition in electronic medical records , 2016, 2016 New York Scientific Data Summit (NYSDS).

[12]  Olof Mogren,et al.  Named Entity Recognition in Swedish Health Records with Character-Based Deep Bidirectional LSTMs , 2016, BioTxtM@COLING 2016.

[13]  Hai Zhao,et al.  A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding , 2015, ArXiv.

[14]  Svetha Venkatesh,et al.  Learning vector representation of medical objects via EMR-driven nonnegative restricted Boltzmann machines (eNRBM) , 2015, J. Biomed. Informatics.

[15]  A Hasman,et al.  Medical narratives in electronic medical records. , 1997, International journal of medical informatics.

[16]  Noémie Elhadad,et al.  Automated methods for the summarization of electronic health records , 2015, J. Am. Medical Informatics Assoc..

[17]  Maryam Habibi,et al.  Deep learning with word embeddings improves biomedical named entity recognition , 2017, Bioinform..

[18]  Nanyun Peng,et al.  Improving Named Entity Recognition for Chinese Social Media with Word Segmentation Representation Learning , 2016, ACL.

[19]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[20]  Hongfei Lin,et al.  ML-CNN: A novel deep learning based disease named entity recognition architecture , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[21]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[22]  Bin Dong,et al.  Building a comprehensive syntactic and semantic corpus of Chinese clinical texts , 2016, J. Biomed. Informatics.

[23]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[24]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[25]  N. Terry,et al.  The Emergence of National Electronic Health Record Architectures in the United States and Australia: Models, Costs, and Questions , 2005, Journal of medical Internet research.

[26]  Ameya Prabhu,et al.  Towards Deep Learning in Hindi NER: An approach to tackle the Labelled Data Sparsity , 2016, ICON.

[27]  Hongfang Liu,et al.  CliniViewer: A Tool for Viewing Electronic Medical Records Based on Natural Language Processing and XML , 2004, MedInfo.

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[30]  Christopher D. Manning,et al.  Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models , 2016, ACL.

[31]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[32]  Donia Scott,et al.  Extracting information from the text of electronic medical records to improve case detection: a systematic review , 2016, J. Am. Medical Informatics Assoc..

[33]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[34]  Paul D. Clayton,et al.  Use and Impact of a Computer-Generated Patient Summary Worksheet for Primary Care , 2005, AMIA.