Attention-Gated Graph Convolutions for Extracting Drug Interaction Information from Drug Labels

Preventable adverse events as a result of medical errors present a growing concern in the healthcare system. As drug-drug interactions (DDIs) may lead to preventable adverse events, being able to extract DDIs from drug labels into a machine-processable form is an important step toward effective dissemination of drug safety information. In this study, we tackle the problem of jointly extracting drugs and their interactions, including interaction outcome, from drug labels. Our deep learning approach entails composing various intermediate representations including sequence and graph based context, where the latter is derived using graph convolutions (GC) with a novel attention-based gating mechanism (holistically called GCA). These representations are then composed in meaningful ways to handle all subtasks jointly. To overcome scarcity in training data, we additionally propose transfer learning by pre-training on related DDI data. Our model is trained and evaluated on the 2018 TAC DDI corpus. Our GCA model in conjunction with transfer learning performs at 39.20% F1 and 26.09% F1 on entity recognition (ER) and relation extraction (RE) respectively on the first official test set and at 45.30% F1 and 27.87% F1 on ER and RE respectively on the second official test set corresponding to an improvement over our prior best results by up to 6 absolute F1 points. After controlling for available training data, our model exhibits state-of-the-art performance by improving over the next comparable best outcome by roughly three F1 points in ER and 1.5 F1 points in RE evaluation across two official test sets.

[1]  Paloma Martínez,et al.  Exploring convolutional neural networks for drug–drug interaction extraction , 2017, Database J. Biol. Databases Curation.

[2]  Bharath Dandala,et al.  IBM Research System at TAC 2018: Deep Learning architectures for Drug-Drug Interaction extraction from Structured Product Labels , 2018, TAC.

[3]  Paloma Martínez,et al.  The DDI corpus: An annotated corpus with pharmacological substances and drug-drug interactions , 2013, J. Biomed. Informatics.

[4]  Burr Settles,et al.  Active Learning , 2012, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[5]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[6]  Tutut Herawan,et al.  Computational and mathematical methods in medicine. , 2006, Computational and mathematical methods in medicine.

[7]  Eric Nichols,et al.  Named Entity Recognition with Bidirectional LSTM-CNNs , 2015, TACL.

[8]  Jaewoo Kang,et al.  Drug drug interaction extraction from the literature using a recursive neural network , 2018, PloS one.

[9]  L. Kohn,et al.  To Err Is Human : Building a Safer Health System , 2007 .

[10]  Fei Li,et al.  A neural joint model for entity and relation extraction from biomedical text , 2017, BMC Bioinformatics.

[11]  Hongfei Lin,et al.  Drug drug interaction extraction from biomedical literature using syntax convolutional neural network , 2016, Bioinform..

[12]  Xiaolong Wang,et al.  Drug-Drug Interaction Extraction via Convolutional Neural Networks , 2016, Comput. Math. Methods Medicine.

[13]  Christopher D. Manning,et al.  Graph Convolution over Pruned Dependency Trees Improves Relation Extraction , 2018, EMNLP.

[14]  Ellen Riloff Bootstrapping for text learning tasks , 1999 .

[15]  Halil Kilicoglu,et al.  A Multi-Task Learning Framework for Extracting Drugs and Their Interactions from Drug Labels , 2019, TAC.

[16]  Tung Tran,et al.  Extracting Drug-Drug Interactions with Word and Character-Level Recurrent Neural Networks , 2017, 2017 IEEE International Conference on Healthcare Informatics (ICHI).

[17]  Makoto Miwa,et al.  Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information , 2018, ACL.

[18]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[19]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[20]  Sunil Kumar Sahu,et al.  Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network , 2017, J. Biomed. Informatics.

[21]  Kai Chen,et al.  Dependency-based convolutional neural network for drug-drug interaction extraction , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[22]  Jaewoo Kang,et al.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining , 2019, Bioinform..

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jun'ichi Tsujii,et al.  GENIA corpus - a semantically annotated corpus for bio-textmining , 2003, ISMB.

[25]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[26]  Dan Roth,et al.  Design Challenges and Misconceptions in Named Entity Recognition , 2009, CoNLL.

[27]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[28]  Travis R. Goodwin,et al.  Overview of the TAC 2018 Drug-Drug Interaction Extraction from Drug Labels Track , 2018, TAC.

[29]  Tapio Salakoski,et al.  Distributional Semantics Resources for Biomedical Text Processing , 2013 .

[30]  Yueting Zhuang,et al.  Two Step Joint Model for Drug Drug Interaction Extraction , 2018, TAC.

[31]  Jun Feng,et al.  Drug-Drug Interaction Extraction via Recurrent Hybrid Convolutional Neural Networks with an Improved Focal Loss , 2019, Entropy.

[32]  David Cohn,et al.  Active Learning , 2010, Encyclopedia of Machine Learning.