Machine learning based early mortality prediction in the emergency department

BACKGROUND It is a great challenge for emergency physicians to early detect the patient's deterioration and prevent unexpected death through a large amount of clinical data, which requires sufficient experience and keen insight. OBJECTIVE To evaluate the performance of machine learning models in quantifying the severity of emergency department (ED) patients and identifying high-risk patients. METHODS Using routinely-available demographics, vital signs and laboratory tests extracted from electronic health records (EHRs), a framework based on machine learning and feature engineering was proposed for mortality prediction. Patients who had one complete record of vital signs and laboratory tests in ED were included. The following patients were excluded: pediatric patients aged < 18 years, pregnant woman, and patients died or were discharged or hospitalized within 12 h after admission. Based on 76 original features extracted, 9 machine learning models were adopted to validate our proposed framework. Their optimal hyper-parameters were fine-tuned using the grid search method. The prediction results were evaluated on performance metrics (i.e., accuracy, area under the curve (AUC), recall and precision) with repeated 5-fold cross-validation (CV). The time window from patient admission to the prediction was analyzed at 12 h, 24 h, 48 h, and entire stay. RESULTS We studied a total of 1114 ED patients with 71.54% (797/1114) survival and 28.46% (317/1114) death in the hospital. The results revealed a more complete time window leads to better prediction performance. Using the entire stay records, the LightGBM model with refined feature engineering demonstrated high discrimination and achieved 93.6% (±0.008) accuracy, 97.6% (±0.003) AUC, 97.1% (±0.008) recall, and 94.2% (±0.006) precision, even if no diagnostic information was utilized. CONCLUSIONS This study quantifies the criticality of ED patients and appears to have significant potential as a clinical decision support tool in assisting physicians in their clinical routine. While the model requires validation before use elsewhere, the same methodology could be used to create a strong model for the new hospital.

[1]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[2]  Mohammad Hassan Moradi,et al.  Mortality prediction in intensive care units (ICUs) using a deep rule-based fuzzy classifier , 2018, J. Biomed. Informatics.

[3]  W. Cha,et al.  Predicting Adverse Outcomes for Febrile Patients in the Emergency Department Using Sparse Laboratory Data: Development of a Time Adaptive Model , 2020, JMIR medical informatics.

[4]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[5]  Alessandra Alaniz Macedo,et al.  Prediction of mortality in Intensive Care Units: a multivariate feature selection , 2020, J. Biomed. Informatics.

[6]  Manuel Graña,et al.  Balanced training of a hybrid ensemble method for imbalanced datasets: a case of emergency department readmission prediction , 2017, Neural Computing and Applications.

[7]  Scott Levin,et al.  Machine‐Learning‐Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index , 2017, Annals of emergency medicine.

[8]  Tadahiro Goto,et al.  Emergency department triage prediction of clinical outcomes using machine learning models , 2019, Critical Care.

[9]  Manuel Graña,et al.  Predictive models for hospital readmission risk: A systematic review of methods , 2018, Comput. Methods Programs Biomed..

[10]  Joao M. C. Sousa,et al.  Clinical Decision Support Systems for Triage in the Emergency Department using Intelligent Systems: a Review , 2020, Artif. Intell. Medicine.

[11]  A. Mahajan,et al.  Machine Learning Prediction of Postoperative Emergency Department Hospital Readmission , 2020, Anesthesiology.

[12]  L. Tarassenko,et al.  Predicting in-hospital mortality and unanticipated admissions to the intensive care unit using routinely collected blood tests and vital signs: Development and validation of a multivariable model☆ , 2018, Resuscitation.

[13]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[14]  Ken Masamune,et al.  Use of Machine-Learning Approaches to Predict Clinical Deterioration in Critically Ill Patients: A Systematic Review , 2017 .

[15]  Yehezkel S. Resheff,et al.  A Gradient Boosting Machine Learning Model for Predicting Early Mortality in the Emergency Department Triage: Devising a Nine-Point Triage Score , 2019, Journal of General Internal Medicine.

[16]  G. Moody,et al.  Predicting in-hospital mortality of ICU patients: The PhysioNet/Computing in cardiology challenge 2012 , 2012, 2012 Computing in Cardiology.

[17]  David J. Stone,et al.  State of the art review: the data revolution in critical care , 2015, Critical Care.

[18]  Bart Vanrumste,et al.  Feature Engineering for ICU Mortality Prediction Based on Hourly to Bi-Hourly Measurements , 2019, Applied Sciences.

[19]  S. Lemeshow,et al.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. , 1993, JAMA.

[20]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[21]  Pierre Baldi,et al.  Development and Validation of a Deep Neural Network Model for Prediction of Postoperative In-hospital Mortality , 2018, Anesthesiology.

[22]  Andreas W. Kempa-Liehr,et al.  Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package) , 2018, Neurocomputing.

[23]  Sungjoo Lee,et al.  Prediction of Cardiac Arrest in the Emergency Department Based on Machine Learning and Sequential Characteristics: Model Development and Retrospective Clinical Validation Study , 2020, JMIR medical informatics.

[24]  M. Keegan,et al.  Severity of illness scoring systems in the intensive care unit , 2011, Critical care medicine.

[25]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[26]  Tudor Toma,et al.  Discovery and inclusion of SOFA score episodes in mortality prediction , 2007, J. Biomed. Informatics.

[27]  S. Kheterpal,et al.  Machine Learning Comes of Age: Local Impact versus National Generalizability. , 2020, Anesthesiology.

[28]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[29]  Lucila Ohno-Machado,et al.  Logistic regression and artificial neural network classification models: a methodology review , 2002, J. Biomed. Informatics.

[30]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[31]  Yeha Lee,et al.  Validation of deep-learning-based triage and acuity score using a large national dataset , 2018, PloS one.

[32]  Roger G. Mark,et al.  Reproducibility in critical care: a mortality prediction case study , 2017, MLHC.

[33]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[34]  William A Knaus,et al.  APACHE 1978-2001: the development of a quality assurance system based on prognosis: milestones and personal reflections. , 2002, Archives of surgery.

[35]  A. Rastegari,et al.  Association of triage time Shock Index, Modified Shock Index, and Age Shock Index with mortality in Emergency Severity Index level 2 patients. , 2016, The American journal of emergency medicine.

[36]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[37]  Karsten M. Borgwardt,et al.  Early prediction of circulatory failure in the intensive care unit using machine learning , 2020, Nature Medicine.

[38]  Chih-Yu Yang,et al.  Prediction of the development of acute kidney injury following cardiac surgery by machine learning , 2020, Critical Care.