Understanding the Artificial Intelligence Clinician and optimal treatment strategies for sepsis in intensive care

In this document, we explore in more detail our published work (Komorowski, Celi, Badawi, Gordon, & Faisal, 2018) for the benefit of the AI in Healthcare research community. In the above paper, we developed the AI Clinician system, which demonstrated how reinforcement learning could be used to make useful recommendations towards optimal treatment decisions from intensive care data. Since publication a number of authors have reviewed our work (e.g. Abbasi, 2018; Bos, Azoulay, & Martin-Loeches, 2019; Saria, 2018). Given the difference of our framework to previous work, the fact that we are bridging two very different academic communities (intensive care and machine learning) and that our work has impact on a number of other areas with more traditional computer-based approaches (biosignal processing and control, biomedical engineering), we are providing here additional details on our recent publication.

[1]  J. Vincent,et al.  A positive fluid balance is an independent prognostic factor in patients with sepsis , 2015, Critical Care.

[2]  C. Permpikul,et al.  Early Use of Norepinephrine in Septic Shock Resuscitation (CENSER). A Randomized Trial , 2019, American journal of respiratory and critical care medicine.

[3]  Lieuwe D J Bos,et al.  Future of the ICU: finding treatable needles in the data haystack , 2018, Intensive Care Medicine.

[4]  Taka-aki Nakada,et al.  Fluid resuscitation in septic shock: A positive fluid balance and elevated central venous pressure are associated with increased mortality* , 2011, Critical care medicine.

[5]  D. Schoenfeld,et al.  Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. , 2000, The New England journal of medicine.

[6]  Nan Jiang,et al.  Doubly Robust Off-policy Value Evaluation for Reinforcement Learning , 2015, ICML.

[7]  Jørn Wetterslev,et al.  Restricting volumes of resuscitation fluid in adults with septic shock after initial management: the CLASSIC randomised, parallel-group, multicentre feasibility trial , 2016, Intensive Care Medicine.

[8]  Adil Rafiq Rather,et al.  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) , 2015 .

[9]  Peter Stone,et al.  Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation , 2016, AAAI.

[10]  Peter Stone,et al.  High Confidence Off-Policy Evaluation with Models , 2016, ArXiv.

[11]  Maxime Cannesson,et al.  Automated Titration of Vasopressor Infusion Using a Closed-loop Controller: In Vivo Feasibility Study Using a Swine Model , 2019, Anesthesiology.

[12]  S. Saria.  Individualized sepsis treatment using reinforcement learning , 2018, Nature Medicine.

[13]  Shamim Nemati,et al.  Does the "Artificial Intelligence Clinician" learn optimal treatment strategies for sepsis in intensive care? , 2019, ArXiv.

[14]  Alistair E. W. Johnson,et al.  The eICU Collaborative Research Database, a freely available multi-center database for critical care research , 2018, Scientific Data.

[15]  Aldo A. Faisal,et al.  The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care , 2018, Nature Medicine.

[16]  Yao Liu,et al.  Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters , 2018, ArXiv.

[17]  Sangeeta Mehta,et al.  Higher versus lower blood pressure targets for vasopressor therapy in shock: a multicentre pilot randomized controlled trial , 2016, Intensive Care Medicine.

[18]  M. Lilot,et al.  Closed-loop assisted versus manual goal-directed fluid therapy during high-risk abdominal surgery: a case–control study with propensity matching , 2015, Critical Care.

[19]  Yao Liu.  Representation Balancing MDPs for Off-Policy Policy Evaluation , 2019 .

[20]  Fredrik D. Johansson,et al.  Guidelines for reinforcement learning in healthcare , 2019, Nature Medicine.

[21]  Matthieu Komorowski,et al.  The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making , 2018, ArXiv.

[22]  S. Goldstein,et al.  Hydroxyethyl Starch or Saline for Fluid Resuscitation in Intensive Care. , 2016, The New England journal of medicine.

[23]  Jennifer Abbasi,et al.  Artificial Intelligence Tools for Sepsis and Cancer. , 2018, JAMA.

[24]  Andrew Slavin Ross,et al.  Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning , 2018, AMIA.

[25]  Philip S. Thomas,et al.  High Confidence Policy Improvement , 2015, ICML.

[26]  F. Machado,et al.  Positive fluid balance as a prognostic factor for mortality and acute kidney injury in severe sepsis and septic shock. , 2015, Journal of critical care.

[27]  Doina Precup,et al.  Eligibility Traces for Off-Policy Policy Evaluation , 2000, ICML.

[28]  Danny McLaughlin,et al.  High versus Low Blood Pressure Target in Patients with Septic Shock , 2014 .