A Dual-Attention Network for Joint Named Entity Recognition and Sentence Classification of Adverse Drug Events

An adverse drug event (ADE) is an injury resulting from medical intervention related to a drug. Automatic ADE detection from text is either fine-grained (ADE entity recognition) or coarse-grained (ADE assertive sentence classification), with limited efforts leveraging inter-dependencies among the two granularities. We instead propose a multi-grained joint deep network to concurrently learn the ADE entity recognition and ADE sentence classification tasks. Our joint approach takes advantage of their symbiotic relationship, with a transfer of knowledge between the two levels of granularity. Our dual-attention mechanism constructs multiple distinct representations of a sentence that capture both task-specific and semantic information in the sentence, providing stronger emphasis on the key elements essential for sentence classification. Our model improves state-of- art F1-score for both tasks: (i) entity recognition of ADE words (12.5% increase) and (ii) ADE sentence classification (13.6% increase) on MADE 1.0 benchmark of EHR notes.

[1]  Hong Yu,et al.  Towards Drug Safety Surveillance and Pharmacovigilance: Current Progress in Detecting Medication and Adverse Drug Events from Electronic Health Records , 2019, Drug Safety.

[2]  Franck Dernoncourt,et al.  Neural Networks for Joint Sentence Classification in Medical Paper Abstracts , 2017, EACL.

[3]  Stefan M. Rüger,et al.  Adverse Drug Reaction Classification With Deep Neural Networks , 2016, COLING.

[4]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[5]  Hong Yu,et al.  Structured prediction models for RNN based sequence labeling in clinical text , 2016, EMNLP.

[6]  Elke A. Rundensteiner,et al.  Adverse Drug Event Detection from Electronic Health Records Using Hierarchical Recurrent Neural Networks with Dual-Level Embedding , 2019, Drug Safety.

[7]  Zina M. Ibrahim,et al.  Improving RNN with Attention and Embedding for Adverse Drug Reactions , 2017, DH.

[8]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[9]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[12]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[13]  Lemao Liu,et al.  Neural Machine Translation with Supervised Attention , 2016, COLING.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[16]  Zi Huang,et al.  Multi-attention Network for One Shot Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yen S. Low,et al.  Text Mining for Adverse Drug Events: the Promise, Challenges, and State of the Art , 2014, Drug Safety.

[18]  Eric R. LaRose,et al.  Adverse Drug Event Discovery Using Biomedical Literature: A Big Data Neural Network Adventure , 2017, JMIR medical informatics.

[19]  Hong Yu,et al.  Bidirectional RNN for Medical Event Detection in Electronic Health Records , 2016, NAACL.

[20]  Anand S. Rao,et al.  Attention-Based Multi-Task Learning in Pharmacovigilance , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[21]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[22]  Anders Søgaard,et al.  Jointly Learning to Label Sentences and Tokens , 2018, AAAI.

[23]  C. Marano,et al.  To err is human. Building a safer health system , 2005 .

[24]  P. Maurette.  [To err is human: building a safer health system]. , 2002, Annales francaises d'anesthesie et de reanimation.

[25]  L. Kohn,et al.  To Err Is Human : Building a Safer Health System , 2007 .

[26]  Hong Yu,et al.  Overview of the First Natural Language Processing Challenge for Extracting Medication, Indication, and Adverse Drug Events from Electronic Health Record Notes (MADE 1.0) , 2019, Drug Safety.

[27]  Kenneth Heafield,et al.  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers , 2016, Annual Meeting of the Association for Computational Linguistics.

[28]  Anand S. Rao,et al.  Automated classification of adverse events in pharmacovigilance , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).