A novel deep learning approach to extract Chinese clinical entities for lung cancer screening and staging

Computed tomography (CT) reports record a large volume of valuable information about patients’ conditions and the interpretations of radiology images from radiologists, which can be used for clinical decision-making and further academic study. However, the free-text nature of clinical reports is a critical barrier to use this data more effectively. In this study, we investigate a novel deep learning method to extract entities from Chinese CT reports for lung cancer screening and TNM staging. The proposed approach presents a new named entity recognition algorithm, namely the BERT-based-BiLSTM-Transformer network (BERT-BTN) with pre-training, to extract clinical entities for lung cancer screening and staging. Specifically, instead of traditional word embedding methods, BERT is applied to learn the deep semantic representations of characters. Following the long short-term memory layer, a Transformer layer is added to capture the global dependencies between characters. Besides, pre-training technique is employed to alleviate the problem of insufficient labeled data. We verify the effectiveness of the proposed approach on a clinical dataset containing 359 CT reports collected from the Department of Thoracic Surgery II of Peking University Cancer Hospital. The experimental results show that the proposed approach achieves an 85.96% macro-F1 score under exact match scheme, which improves the performance by 1.38%, 1.84%, 3.81%,4.29%,5.12%,5.29% and 8.84% compared to BERT-BTN, BERT-LSTM, BERT-fine-tune, BERT-Transformer, FastText-BTN, FastText-BiLSTM and FastText-Transformer, respectively. In this study, we developed a novel deep learning method, i.e., BERT-BTN with pre-training, to extract the clinical entities from Chinese CT reports. The experimental results indicate that the proposed approach can efficiently recognize various clinical entities about lung cancer screening and staging, which shows the potential for further clinical decision-making and academic research.

[1]  Jingqi Wang,et al.  Enhancing Clinical Concept Extraction with Contextual Embedding , 2019, J. Am. Medical Informatics Assoc..

[2]  Saeed Hassanpour,et al.  Artificial Intelligence in Medicine , 2015 .

[3]  Bin Wang,et al.  Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy , 2020, ECAI.

[4]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[5]  M. Levy,et al.  ReCAP: Feasibility and Accuracy of Extracting Cancer Stage Information From Narrative Electronic Health Record Data. , 2016, Journal of oncology practice.

[6]  Mourad Gridach,et al.  Character-level neural network for biomedical named entity recognition , 2017, J. Biomed. Informatics.

[7]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[8]  Yue Zhang,et al.  Joint Extraction of Entities and Relations Based on a Novel Graph Scheme , 2018, IJCAI.

[9]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[10]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[11]  Katharine A. Rendle,et al.  The Importance Of Integrating Narrative Into Health Care Decision Making. , 2016, Health affairs.

[12]  Wei-Yun Ma,et al.  Understanding and Improving Sequence-Labeling NER with Self-Attentive LSTMs , 2018 .

[13]  Shang Gao,et al.  Hierarchical attention networks for information extraction from cancer pathology reports , 2017, J. Am. Medical Informatics Assoc..

[14]  Lutz Prechelt,et al.  Automatic early stopping using cross validation: quantifying the criteria , 1998, Neural Networks.

[15]  Brian T. Denton,et al.  Clinical predictors and recommendations for staging computed tomography scan among men with prostate cancer. , 2014, Urology.

[16]  Alan R. Aronson,et al.  The NLM Indexing Initiative: Current Status and Role in Improving Access to Biomedical Information , 1998 .

[17]  Huilong Duan,et al.  Utilizing Chinese Admission Records for MACE Prediction of Acute Coronary Syndrome , 2016, International journal of environmental research and public health.

[18]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[19]  Liangli Ma,et al.  Combined Self-Attention Mechanism for Chinese Named Entity Recognition in Military , 2019, Future Internet.

[20]  Liang Chen,et al.  Using natural language processing to extract clinically useful information from Chinese electronic medical records , 2019, Int. J. Medical Informatics.

[21]  Tong Zhang,et al.  Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings , 2016, ICML.

[22]  Khalid Ashraf,et al.  Pathology Extraction from Chest X-Ray Radiology Reports: A Performance Study , 2018, ArXiv.

[23]  Nan Wu,et al.  Predicting postoperative non-small cell lung cancer prognosis via long short-term relational regularization , 2020, Artif. Intell. Medicine.

[24]  George Hripcsak,et al.  Technical Brief: Agreement, the F-Measure, and Reliability in Information Retrieval , 2005, J. Am. Medical Informatics Assoc..

[25]  Ergin Soysal,et al.  Identifying Metastases-related Information from Pathology Reports of Lung Cancer Patients , 2017, CRI.

[26]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[27]  Hua Xu,et al.  Research and applications: Assisted annotation of medical free text using RapTAT , 2014, J. Am. Medical Informatics Assoc..

[28]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[29]  Wanxiang Che,et al.  Pre-Training with Whole Word Masking for Chinese BERT , 2019, ArXiv.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Hong Gu,et al.  Clinically Significant Information Extraction from Radiology Reports , 2017, DocEng.

[32]  Anthony N. Nguyen,et al.  Symbolic rule-based classification of lung cancer stages from free-text pathology reports , 2010, J. Am. Medical Informatics Assoc..

[33]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[34]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[35]  Chengyi Zheng,et al.  Extracting data from electronic medical records: validation of a natural language processing program to assess prostate biopsy results , 2013, World Journal of Urology.

[36]  Zhiwei Wang,et al.  R-Transformer: Recurrent Neural Network Enhanced Transformer , 2019, ArXiv.

[37]  Rico Sennrich,et al.  Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures , 2018, EMNLP.

[38]  Shun Lu,et al.  Retrospect and Prospect for Lung Cancer in China: Clinical Advances of Immune Checkpoint Inhibitors , 2019, The oncologist.

[39]  Matthias Grabmair,et al.  Supervised Contextual Embeddings for Transfer Learning in Natural Language Processing Tasks , 2019, ArXiv.

[40]  R. Rami-Porta,et al.  The Eighth Edition of the Tumor, Node, and Metastasis Classification of Lung Cancer , 2018 .

[41]  Graciela Gonzalez-Hernandez,et al.  Clinical NER and Relation Extraction using Bi-Char-LSTMs and Random Forest Classifiers , 2018, Medication and Adverse Drug Event Detection.

[42]  S. Waqar Jaffry,et al.  Information extraction from scientific articles: a survey , 2018, Scientometrics.

[43]  John F. Hurdle,et al.  Extracting Information from Textual Documents in the Electronic Health Record: A Review of Recent Research , 2008, Yearbook of Medical Informatics.

[44]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[45]  Christopher C. Yang,et al.  A Framework for Developing and Evaluating Word Embeddings of Drug-named Entity , 2018, BioNLP.

[46]  Yu Zhang,et al.  Clinical Named Entity Recognition From Chinese Electronic Health Records via Machine Learning Methods , 2018, JMIR medical informatics.