Applying Self-interaction Attention for Extracting Drug-Drug Interactions

Discovering the effect of the simultaneous assumption of drugs is a very important field in medical research that could improve the effectiveness of healthcare and avoid adverse drug reactions which can cause health problems to patients. Although there are several pharmacological databases containing information on drugs, this type of information is often expressed in the form of free text. Analyzing sentences in order to extract drug-drug interactions was the objective of the DDIExtraction-2013 task. Despite the fact that the challenge took place six years ago, the interest on this task is still active and several new methods based on Recurrent Neural Networks and Attention Mechanisms have been designed. In this paper, we propose a model that combines bidirectional Long Short Term Memory (LSTM) networks with the Self-Interaction Attention Mechanism. Experimental analysis shows how this model improves the classification accuracy reducing the tendency to predict the majority class resulting in false negatives, over several input configurations.

[1]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[2]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[3]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[4]  Shasha Li,et al.  Drug-Drug Interaction Extraction via Recurrent Neural Network with Multiple Attention Layers , 2017, ADMA.

[5]  Ion Androutsopoulos,et al.  Deep Relevance Ranking Using Enhanced Document-Query Interactions , 2018, EMNLP.

[6]  Paloma Martínez,et al.  Lessons learnt from the DDIExtraction-2013 Shared Task , 2014, J. Biomed. Informatics.

[7]  Sunil Kumar Sahu,et al.  Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network , 2017, J. Biomed. Informatics.

[8]  Alberto Lavelli,et al.  FBK-irst : A Multi-Phase Kernel Based Approach for Drug-Drug Interaction Detection and Classification that Exploits Linguistic Information , 2013, *SEMEVAL.

[9]  Kanchan M.Tarwani,et al.  Survey on Recurrent Neural Network in Natural Language Processing , 2017 .

[10]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[11]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[12]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[13]  Yijia Zhang,et al.  An attention-based effective neural model for drug-drug interactions extraction , 2017, BMC Bioinformatics.

[14]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[15]  Foster Provost,et al.  The effect of class distribution on classifier learning: an empirical study , 2001 .

[16]  Wei Zheng,et al.  Drug–drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths , 2017, Bioinform..

[17]  Andy Way,et al.  Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction , 2018, EMNLP.

[18]  Jari Björne,et al.  UTurku: Drug Named Entity Recognition and Drug-Drug Interaction Extraction Using SVM Classification and Domain Knowledge , 2013, SemEval@NAACL-HLT.

[19]  Isabel Segura-Bedmar,et al.  Evaluation of pooling operations in convolutional architectures for drug-drug interaction extraction , 2018, BMC Bioinformatics.

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[22]  Lishuang Li,et al.  An End-to-End Entity and Relation Extraction Network with Multi-head Attention , 2018, CCL.

[23]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[24]  Jianming Zheng,et al.  Self-Interaction Attention Mechanism-Based Text Representation for Document Classification , 2018 .

[25]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[26]  Christopher D. Manning,et al.  Graph Convolution over Pruned Dependency Trees Improves Relation Extraction , 2018, EMNLP.

[27]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[28]  Xiao Sun,et al.  Multichannel Convolutional Neural Network for Biological Relation Extraction , 2016, BioMed research international.

[29]  Xiaolong Wang,et al.  Drug-Drug Interaction Extraction via Convolutional Neural Networks , 2016, Comput. Math. Methods Medicine.

[30]  Jaewoo Kang,et al.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining , 2019, Bioinform..

[31]  Tao Yang,et al.  Word Embedding for Understanding Natural Language: A Survey , 2018 .

[32]  Rudolf Kadlec,et al.  Text Understanding with the Attention Sum Reader Network , 2016, ACL.

[33]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.