A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice.

OBJECTIVES Develop and implement a machine learning algorithm to predict severe sepsis and septic shock and evaluate the impact on clinical practice and patient outcomes. DESIGN Retrospective cohort for algorithm derivation and validation, pre-post impact evaluation. SETTING Tertiary teaching hospital system in Philadelphia, PA. PATIENTS All non-ICU admissions; algorithm derivation July 2011 to June 2014 (n = 162,212); algorithm validation October to December 2015 (n = 10,448); silent versus alert comparison January 2016 to February 2017 (silent n = 22,280; alert n = 32,184). INTERVENTIONS A random-forest classifier, derived and validated using electronic health record data, was deployed both silently and later with an alert to notify clinical teams of sepsis prediction. MEASUREMENT AND MAIN RESULT Patients identified for training the algorithm were required to have International Classification of Diseases, 9th Edition codes for severe sepsis or septic shock and a positive blood culture during their hospital encounter with either a lactate greater than 2.2 mmol/L or a systolic blood pressure less than 90 mm Hg. The algorithm demonstrated a sensitivity of 26% and specificity of 98%, with a positive predictive value of 29% and positive likelihood ratio of 13. The alert resulted in a small statistically significant increase in lactate testing and IV fluid administration. There was no significant difference in mortality, discharge disposition, or transfer to ICU, although there was a reduction in time-to-ICU transfer. CONCLUSIONS Our machine learning algorithm can predict, with low sensitivity but high specificity, the impending occurrence of severe sepsis and septic shock. Algorithm-generated predictive alerts modestly impacted clinical measures. Next steps include describing clinical perception of this tool and optimizing algorithm design and delivery.

[1]  Michael Bailey,et al.  A controlled trial of electronic automated advisory vital signs monitoring in general hospital wards* , 2012, Critical care medicine.

[2]  Joel S. Betesh,et al.  Clinician Perception of the Effectiveness of an Automated Early Warning and Response System for Sepsis in an Academic Medical Center. , 2015, Annals of the American Thoracic Society.

[3]  Jessica S. Ancker,et al.  Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system , 2017, BMC Medical Informatics and Decision Making.

[4]  David O. Meltzer,et al.  Multicenter Comparison of Machine Learning Methods and Conventional Regression for Predicting Clinical Deterioration on the Wards , 2016, Critical care medicine.

[5]  Yixin Chen,et al.  Toward a two-tier clinical warning system for hospitalized patients. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[6]  Ben J. Marafino,et al.  Validation of Prediction Models for Critical Care Outcomes Using Natural Language Processing of Electronic Health Record Data , 2018, JAMA network open.

[7]  Patricia Kipnis,et al.  Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. , 2012, Journal of hospital medicine.

[8]  Susan Gruber,et al.  Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014 , 2017, JAMA.

[9]  F. Cabitza,et al.  Unintended Consequences of Machine Learning in Medicine , 2017, JAMA.

[10]  Sairam Parthasarathy,et al.  Real-Time Automated Sampling of Electronic Medical Records Predicts Hospital Mortality. , 2016, The American journal of medicine.

[11]  Steven Horng,et al.  Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning , 2017, PloS one.

[12]  Ritankar Das,et al.  Reducing patient mortality, length of stay and readmissions through machine learning-based sepsis prediction in the emergency department, intensive care unit and hospital floor units , 2017, BMJ open quality.

[13]  Ben Wellner,et al.  Predicting Unplanned Transfers to the Intensive Care Unit: A Machine Learning Approach Leveraging Diverse Clinical Elements , 2017, JMIR medical informatics.

[14]  R. Bellomo,et al.  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). , 2016, JAMA.

[15]  Ritankar Das,et al.  Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial , 2017, BMJ Open Respiratory Research.

[16]  C. Torio,et al.  National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2011 , 2013 .

[17]  C. Winslow,et al.  Multicenter development and validation of a risk stratification tool for ward patients. , 2014, American journal of respiratory and critical care medicine.

[18]  G. Escobar,et al.  The Timing of Early Antibiotics and Hospital Mortality in Sepsis , 2017, American journal of respiratory and critical care medicine.

[19]  Dean F Sittig,et al.  Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis. , 2012, The American journal of medicine.

[20]  Benjamin French,et al.  Development, implementation, and impact of an automated early warning and response system for sepsis. , 2015, Journal of hospital medicine.

[21]  Michael Draugelis,et al.  Clinician Perception of a Machine Learning-Based Early Warning System Designed to Predict Severe Sepsis and Septic Shock. , 2019, Critical care medicine.

[22]  G. Kuperman,et al.  A Computerized Alert Screening For Severe Sepsis In Emergency Department Patients Increases Lactate Testing But Does Not Improve Inpatient Mortality , 2010, Applied Clinical Informatics.

[23]  Uli K. Chettipally,et al.  Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU , 2018, BMJ Open.

[24]  P. Pronovost,et al.  A targeted real-time early warning score (TREWScore) for septic shock , 2015, Science Translational Medicine.

[25]  Xiaopeng Zhao,et al.  Prediction of ICU in-hospital mortality using a deep Boltzmann machine and dropout neural net , 2013, 2013 Biomedical Sciences and Engineering Conference (BSEC).

[26]  L. Ungar,et al.  Inclusion of Unstructured Clinical Text Improves Early Prediction of Death or Prolonged ICU Stay* , 2018, Critical care medicine.

[27]  K. Borgwardt,et al.  Machine Learning in Medicine , 2015, Mach. Learn. under Resour. Constraints Vol. 3.

[28]  William Fleischman,et al.  Prediction of In-hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data-Driven, Machine Learning Approach. , 2016, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.