Order-free Medicine Combination Prediction with Graph Convolutional Reinforcement Learning

Medicine Combination Prediction (MCP) based on Electronic Health Record (EHR) can assist doctors to prescribe medicines for complex patients. Previous studies on MCP either ignore the correlations between medicines (i.e., MCP is formulated as a binary classifcation task), or assume that there is a sequential correlation between medicines (i.e., MCP is formulated as a sequence prediction task). The latter is unreasonable because the correlations between medicines should be considered in an order-free way. Importantly, MCP must take additional medical knowledge (e.g., Drug-Drug Interaction (DDI)) into consideration to ensure the safety of medicine combinations. However, most previous methods for MCP incorporate DDI knowledge with a post-processing scheme, which might undermine the integrity of proposed medicine combinations. In this paper, we propose a graph convolutional reinforcement learning model for MCP, named Combined Order-free Medicine Prediction Network (CompNet), that addresses the issues listed above. CompNet casts the MCP task as an order-free Markov Decision Process (MDP) problem and designs a Deep Q Learning (DQL) mechanism to learn correlative and adverse interactions between medicines. Specifcally, we frst use a Dual Convolutional Neural Network (Dual-CNN) to obtain patient representations based on EHRs. Then, we introduce the medicine knowledge associated with predicted medicines to create a dynamic medicine knowledge graph, and use a Relational Graph Convolutional Network (R-GCN) to encode it. Finally, CompNet selects medicines by fusing the combination of patient information and the medicine knowledge graph. Experiments on a benchmark dataset, i.e., MIMIC-III, demonstrate that CompNet signifcantly outperforms state-of-the-art methods and improves a recently proposed model by 3.74%pt, 6.64%pt in terms of Jaccard and F1 metrics.

[1]  Yixin Chen,et al.  An End-to-End Deep Learning Architecture for Graph Classification , 2018, AAAI.

[2]  C. Krittanawong,et al.  The rise of artificial intelligence and the uncertain future for physicians. , 2017, European journal of internal medicine.

[3]  Thomas A. Lasko,et al.  Predicting Medications from Diagnostic Codes with Recurrent Neural Networks , 2016, ICLR.

[4]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[5]  Lu Wang,et al.  Personalized Prescription for Comorbidity , 2018, DASFAA.

[6]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[7]  Cheng Li,et al.  Conditional Bernoulli Mixtures for Multi-label Classification , 2016, ICML.

[8]  Jure Leskovec,et al.  Modeling polypharmacy side effects with graph convolutional networks , 2018, bioRxiv.

[9]  Wei Wu,et al.  SGM: Sequence Generation Model for Multi-label Classification , 2018, COLING.

[10]  R. Altman,et al.  Data-Driven Prediction of Drug Effects and Interactions , 2012, Science Translational Medicine.

[11]  Wei Dong,et al.  Effect of statin use within the first 24 hours of admission for acute myocardial infarction on early morbidity and mortality. , 2005, The American journal of cardiology.

[12]  Dejia Shi,et al.  Using a LSTM-RNN Based Deep Learning Framework for ICU Mortality Prediction , 2018, WISA.

[13]  Olivier Gevaert,et al.  MicroRNA based Pan-Cancer Diagnosis and Treatment Recommendation , 2016, BMC Bioinformatics.

[14]  Matthieu Komorowski,et al.  Model-Based Reinforcement Learning for Sepsis Treatment , 2018, ArXiv.

[15]  M. de Rijke,et al.  RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation , 2018, AAAI.

[16]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[17]  Meng Wu,et al.  Knowledge Guided Multi-instance Multi-label Learning via Neural Networks in Medicines Prediction , 2018, ACML.

[18]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[19]  Sven Kosub,et al.  A note on the triangle inequality for the Jaccard distance , 2016, Pattern Recognit. Lett..

[20]  V. Dzau,et al.  Health and societal implications of medical and technological advances , 2018, Science Translational Medicine.

[21]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[22]  Fei Wang,et al.  Drug Similarity Integration Through Attentive Multi-view Graph Auto-Encoders , 2018, IJCAI.

[23]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[24]  Shamim Nemati,et al.  Optimal medication dosing from suboptimal clinical examples: A deep reinforcement learning approach , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[25]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[26]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[27]  Mehdi Jafari,et al.  National registry of myocardial infarction , 2016 .

[28]  Nahum Shimkin,et al.  Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning , 2016, ICML.

[29]  M. W Gardner,et al.  Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences , 1998 .

[30]  Jimeng Sun,et al.  Explainable Prediction of Medical Codes from Clinical Text , 2018, NAACL.

[31]  Jimeng Sun,et al.  GAMENet: Graph Augmented MEmory Networks for Recommending Medication Combination , 2018, AAAI.

[32]  Albert-László Barabási,et al.  Network-based prediction of drug combinations , 2019, Nature Communications.

[33]  T G Wolfsberg,et al.  ADAM, a novel family of membrane proteins containing A Disintegrin And Metalloprotease domain: multipotential functions in cell-cell and cell- matrix interactions , 1995, The Journal of cell biology.

[34]  Leslie Citrome,et al.  Quantifying risk: the role of absolute and relative measures in interpreting risk of adverse reactions from product labels of antipsychotic medications. , 2009, Current drug safety.

[35]  Kai-Fu Tang,et al.  Inquire and Diagnose : Neural Symptom Checking Ensemble using Deep Reinforcement Learning , 2016 .

[36]  Michel Tokic.  Adaptive ε-greedy Exploration in Reinforcement Learning Based on Value Differences , 2010 .

[37]  Pengtao Xie,et al.  Multimodal Machine Learning for Automated ICD Coding , 2018, MLHC.

[38]  Yuan Luo,et al.  MedGCN: Graph Convolutional Networks for Multiple Medical Tasks , 2019, ArXiv.

[39]  Jimeng Sun,et al.  LEAP: Learning to Prescribe Effective and Safe Treatment Combinations for Multimorbidity , 2017, KDD.

[40]  Dacheng Tao,et al.  Reinforced Multi-Label Image Classification by Exploring Curriculum , 2018, AAAI.

[41]  John F Potter,et al.  Lisinopril for the treatment of hypertension within the first 24 hours of acute ischemic stroke and follow-up. , 2007, American journal of hypertension.

[42]  Stéphanie Allassonnière,et al.  A Model-Based Reinforcement Learning Approach for a Rare Disease Diagnostic Task , 2018, ArXiv.

[43]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[44]  Yongfeng Huang,et al.  Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Network , 2018, Scientific Reports.

[45]  Dale M. Needham,et al.  Hospital mortality prediction for intermediate care patients: Assessing the generalizability of the Intermediate Care Unit Severity Score (IMCUSS) , 2018, Journal of critical care.

[46]  Svetha Venkatesh,et al.  Dual Memory Neural Computer for Asynchronous Two-view Sequential Learning , 2018, KDD.

[47]  Edward Y. Chang,et al.  Context-Aware Symptom Checking for Disease Diagnosis Using Hierarchical Reinforcement Learning , 2018, AAAI.

[48]  Meng Wang,et al.  Safe Medicine Recommendation via Medical Knowledge Graph Embedding , 2017, ArXiv.

[49]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[50]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[51]  M. de Rijke,et al.  Sentence Relations for Extractive Summarization with Deep Neural Networks , 2018, ACM Trans. Inf. Syst..