Deep Reinforcement Learning for Sepsis Treatment

Sepsis is a leading cause of mortality in intensive care units and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we propose an approach to deduce treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Our model learns clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. The learned policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.

[1]  C. Sprung,et al.  Sepsis in European intensive care units: Results of the SOAP study* , 2006, Critical care medicine.

[2]  P. Marik,et al.  The demise of early goal‐directed therapy for severe sepsis and septic shock , 2015, Acta anaesthesiologica Scandinavica.

[3]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[4]  S. Opal,et al.  Sepsis: a roadmap for future research. , 2015, The Lancet. Infectious diseases.

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[9]  M. Müllner,et al.  Vasopressors for shock. , 2004, The Cochrane database of systematic reviews.

[10]  R. Bellomo,et al.  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). , 2016, JAMA.

[11]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Sangeeta Mehta,et al.  Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016 , 2017, Intensive Care Medicine.

[14]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[15]  Nan Jiang,et al.  Doubly Robust Off-policy Evaluation for Reinforcement Learning , 2015, ArXiv.

[16]  C. Sprung,et al.  Surviving Sepsis Campaign: International Guidelines for Management of Severe Sepsis and Septic Shock, 2012 , 2013, Intensive Care Medicine.

[17]  Nan Jiang,et al.  Doubly Robust Off-policy Value Evaluation for Reinforcement Learning , 2015, ICML.