Deep learning for drug-drug interaction extraction from the literature: a review

Drug-drug interactions (DDIs) are crucial for drug research and pharmacovigilance. These interactions may cause adverse drug effects that threaten public health and patient safety. Therefore, the DDIs extraction from biomedical literature has been widely studied and emphasized in modern biomedical research. The previous rules-based and machine learning approaches rely on tedious feature engineering, which is labourious, time-consuming and unsatisfactory. With the development of deep learning technologies, this problem is alleviated by learning feature representations automatically. Here, we review the recent deep learning methods that have been applied to the extraction of DDIs from biomedical literature. We describe each method briefly and compare its performance in the DDI corpus systematically. Next, we summarize the advantages and disadvantages of these deep learning models for this task. Furthermore, we discuss some challenges and future perspectives of DDI extraction via deep learning methods. This review aims to serve as a useful guide for interested researchers to further advance bioinformatics algorithms for DDIs extraction from the literature.

[1]  Stephen Fowler,et al.  In Vitro Evaluation of Reversible and Irreversible Cytochrome P450 Inhibition: Current Status on Methodologies and their Utility for Predicting Drug–Drug Interactions , 2008, The AAPS Journal.

[2]  B. Stricker,et al.  Hospitalisations and emergency department visits due to drug–drug interactions: a literature review , 2007, Pharmacoepidemiology and drug safety.

[3]  J. Aronson,et al.  Classifying drug interactions. , 2004, British journal of clinical pharmacology.

[4]  Wei Zheng,et al.  Leveraging Biomedical Resources in Bi-LSTM for Drug-Drug Interaction Extraction , 2018, IEEE Access.

[5]  Nanyun Peng,et al.  Cross-Sentence N-ary Relation Extraction with Graph LSTMs , 2017, TACL.

[6]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[7]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[8]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[9]  Jieping Ye,et al.  Deep convolutional neural networks for annotating gene expression patterns in the mouse brain , 2015, BMC Bioinformatics.

[10]  David S. Wishart,et al.  DrugBank 4.0: shedding new light on drug metabolism , 2013, Nucleic Acids Res..

[11]  Jaewoo Kang,et al.  Drug drug interaction extraction from the literature using a recursive neural network , 2018, PloS one.

[12]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[13]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[14]  M. Afilalo,et al.  Polypharmacy, adverse drug-related events, and potential adverse drug interactions in elderly patients presenting to an emergency department. , 2001, Annals of emergency medicine.

[15]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[16]  Claire Cardie,et al.  Going out on a limb: Joint Extraction of Entity Mentions and Relations without Dependency Trees , 2017, ACL.

[17]  Hongfei Lin,et al.  Drug drug interaction extraction from biomedical literature using syntax convolutional neural network , 2016, Bioinform..

[18]  Haibin Liu,et al.  Extracting drug-drug interactions from literature using a rich feature-based linear kernel approach , 2015, AMIA.

[19]  Yijia Zhang,et al.  An attention-based effective neural model for drug-drug interactions extraction , 2017, BMC Bioinformatics.

[20]  Xiao Li,et al.  Machine Learning Paradigms for Speech Recognition: An Overview , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Michelle Chen,et al.  A Model for Spheroid versus Monolayer Response of SK-N-SH Neuroblastoma Cells to Treatment with 15-Deoxy-PGJ 2 , 2016, Comput. Math. Methods Medicine.

[22]  Yoshua Bengio,et al.  Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks , 2015, IEEE Transactions on Multimedia.

[23]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[24]  Wei Zheng,et al.  Drug–drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths , 2017, Bioinform..

[25]  Peter M. A. Sloot,et al.  A novel feature-based approach to extract drug-drug interactions from biomedical text , 2014, Bioinform..

[26]  Guodong Zhou,et al.  Tree kernel-based semantic relation extraction with rich syntactic and semantic information , 2010, Inf. Sci..

[27]  Tal Lorberbaum,et al.  Coupling Data Mining and Laboratory Experiments to Discover Drug Interactions Causing QT Prolongation. , 2016, Journal of the American College of Cardiology.

[28]  Tapio Salakoski,et al.  Distributional Semantics Resources for Biomedical Text Processing , 2013 .

[29]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[30]  Zhiyuan Liu,et al.  Hierarchical Relation Extraction with Coarse-to-Fine Grained Attention , 2018, EMNLP.

[31]  Hoifung Poon,et al.  Distant Supervision for Cancer Pathway Extraction from Text , 2014, Pacific Symposium on Biocomputing.

[32]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[33]  Walter E. Haefeli,et al.  Information deficits in the summary of product characteristics preclude an optimal management of drug interactions: a comparison with evidence from the literature , 2005, European Journal of Clinical Pharmacology.

[34]  H. Robbins A Stochastic Approximation Method , 1951 .

[35]  Thomas Demeester,et al.  Adversarial training for multi-context joint entity and relation extraction , 2018, EMNLP.

[36]  J K Aronson Drug interactions-information, education, and the British National Formulary. , 2004, British journal of clinical pharmacology.

[37]  Zhenchao Jiang,et al.  Drug-drug interaction extraction from biomedical literature using support vector machine and long short term memory networks , 2017, Inf. Sci..

[38]  Sunil Kumar Sahu,et al.  Drug-Drug Interaction Extraction from Biomedical Text Using Long Short Term Memory Network , 2017, J. Biomed. Informatics.

[39]  Paloma Martínez,et al.  The DDI corpus: An annotated corpus with pharmacological substances and drug-drug interactions , 2013, J. Biomed. Informatics.

[40]  Paolo Rosso,et al.  Drug-Drug Interaction Detection: A New Approach Based on Maximal Frequent Sequences , 2010, Proces. del Leng. Natural.

[41]  Felix Hammann,et al.  Data mining for potential adverse drug–drug interactions , 2014, Expert opinion on drug metabolism & toxicology.

[42]  Zhi Jin,et al.  Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths , 2015, EMNLP.

[43]  Satoshi Sekine,et al.  Preemptive Information Extraction using Unrestricted Relation Discovery , 2006, NAACL.

[44]  P. Jorens,et al.  Exposure of the elderly to potential nephrotoxic drug combinations in Belgium , 2008, Pharmacoepidemiology and drug safety.

[45]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[46]  Shasha Li,et al.  Drug-Drug Interaction Extraction via Recurrent Neural Network with Multiple Attention Layers , 2017, ADMA.

[47]  Zhiyuan Liu,et al.  Neural Relation Extraction with Selective Attention over Instances , 2016, ACL.

[48]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[49]  T Molina,et al.  Quality of interaction database management systems , 2009, Farmacia Hospitalaria (English Edition).

[50]  Jun'ichi Tsujii,et al.  Feature Forest Models for Probabilistic HPSG Parsing , 2008, CL.

[51]  Philippe Cudré-Mauroux,et al.  Relation Extraction Using Distant Supervision , 2018, ACM Comput. Surv..

[52]  Richard B. Berlin,et al.  Predicting adverse drug events from personal health messages. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[53]  Zhiyuan Liu,et al.  FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation , 2018, EMNLP.

[54]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[55]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[56]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[57]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[58]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[59]  R. Altman,et al.  Informatics confronts drug-drug interactions. , 2013, Trends in pharmacological sciences.

[60]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[61]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[62]  R. Altman,et al.  Data-Driven Prediction of Drug Effects and Interactions , 2012, Science Translational Medicine.

[63]  Wei Wang,et al.  Dependency-based long short term memory network for drug-drug interaction extraction , 2017, BMC Bioinformatics.

[64]  Dmitry Zelenko,et al.  Kernel methods for relation extraction , 2003 .

[65]  Makoto Miwa,et al.  Extracting Drug-Drug Interactions with Attention CNNs , 2017, BioNLP.

[66]  R. Lipton,et al.  Assessment of potential drug-drug interactions with a prescription claims database. , 2005, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[67]  Yue Zhang,et al.  N-ary Relation Extraction using Graph-State LSTM , 2018, EMNLP.

[68]  Isabel Segura-Bedmar,et al.  The 1st DDIExtraction-2011 challenge task: Extraction of Drug-Drug Interactions from biomedical texts , 2011 .

[69]  A. Rodríguez-Terol,et al.  Calidad estructural de las bases de datos de interacciones , 2009 .

[70]  Xiao Sun,et al.  Multichannel Convolutional Neural Network for Biological Relation Extraction , 2016, BioMed research international.

[71]  Shiew-Mei Huang,et al.  Predicting Drug–Drug Interactions: An FDA Perspective , 2009, The AAPS Journal.

[72]  Doheon Lee,et al.  Predicting Pharmacodynamic Drug-Drug Interactions through Signaling Propagation Interference on Protein-Protein Interaction Networks , 2015, PloS one.

[73]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[74]  Abeed Sarker,et al.  Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features , 2015, J. Am. Medical Informatics Assoc..

[75]  Rita Businaro,et al.  Why We Need an Efficient and Careful Pharmacovigilance , 2013 .

[76]  Xiaolong Wang,et al.  Drug-Drug Interaction Extraction via Convolutional Neural Networks , 2016, Comput. Math. Methods Medicine.

[77]  Göran Petersson,et al.  Erratum : Potential drug interactions during a three-decade study , 2007 .

[78]  Mitja Lainscak,et al.  Drug-drug interaction software in clinical practice: a systematic review , 2014, European Journal of Clinical Pharmacology.

[79]  Guodong Zhou,et al.  Bilingual Active Learning for Relation Classification via Pseudo Parallel Corpora , 2014, ACL.

[80]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[81]  Isabel Segura-Bedmar,et al.  Evaluation of pooling operations in convolutional architectures for drug-drug interaction extraction , 2018, BMC Bioinformatics.

[82]  David Sontag,et al.  Learning Low-Dimensional Representations of Medical Concepts , 2016, CRI.

[83]  Peter Stott Stockley's drug interactions , 2004 .

[84]  Jian Zhang,et al.  SQuAD: 100,000+ Questions for Machine Comprehension of Text , 2016, EMNLP.

[85]  Olivier Bodenreider,et al.  The Unified Medical Language System (UMLS): integrating biomedical terminology , 2004, Nucleic Acids Res..

[86]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[87]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[88]  Oren Etzioni,et al.  Modeling Missing Data in Distant Supervision for Information Extraction , 2013, TACL.

[89]  Peter G. M. van der Heijden,et al.  On the assessment of adverse drug reactions from spontaneous reporting systems: the influence of under-reporting on odds ratios , 2002 .

[90]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[91]  Wei Zhang,et al.  Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding , 2018, EMNLP.

[92]  R. Manfredini,et al.  Unravelling the Complexity of Inherited Retinal Dystrophies Molecular Testing: Added Value of Targeted Next-Generation Sequencing , 2016, BioMed research international.

[93]  Göran Petersson,et al.  Potential drug interactions during a three-decade study period: a cross-sectional study of a prescription register , 2007, European Journal of Clinical Pharmacology.

[94]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[95]  Doug Downey,et al.  Unsupervised named-entity extraction from the Web: An experimental study , 2005, Artif. Intell..

[96]  Jun Zhao,et al.  Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks , 2015, EMNLP.

[97]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[98]  Herrero-ZazoMaría,et al.  The DDI corpus , 2013 .

[99]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[100]  Zeeshan Ahmed,et al.  Mining biomedical images towards valuable information retrieval in biomedical and life sciences , 2016, Database J. Biol. Databases Curation.

[101]  Vernon M Chinchilli,et al.  Quality of evidence in drug compendia supporting off-label use of typical and atypical antipsychotic medications. , 2012, The International journal of risk & safety in medicine.

[102]  Carol Friedman,et al.  State of the art and development of a drug-drug interaction large scale predictor based on 3D pharmacophoric similarity. , 2014, Current drug metabolism.

[103]  Makoto Miwa,et al.  Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information , 2018, ACL.

[104]  Isabel Segura Bedmar,et al.  Application of information extraction techniques to pharmacological domain : extracting drug-drug interactions , 2011 .

[105]  Xia Sun,et al.  Deep Convolution Neural Networks for Drug-Drug Interaction Extraction , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).