Computerized prediction of intensive care unit discharge after cardiac surgery: development and validation of a Gaussian processes model

BackgroundThe intensive care unit (ICU) length of stay (LOS) of patients undergoing cardiac surgery may vary considerably, and is often difficult to predict within the first hours after admission. The early clinical evolution of a cardiac surgery patient might be predictive for his LOS. The purpose of the present study was to develop a predictive model for ICU discharge after non-emergency cardiac surgery, by analyzing the first 4 hours of data in the computerized medical record of these patients with Gaussian processes (GP), a machine learning technique.MethodsNon-interventional study. Predictive modeling, separate development (n = 461) and validation (n = 499) cohort. GP models were developed to predict the probability of ICU discharge the day after surgery (classification task), and to predict the day of ICU discharge as a discrete variable (regression task). GP predictions were compared with predictions by EuroSCORE, nurses and physicians. The classification task was evaluated using aROC for discrimination, and Brier Score, Brier Score Scaled, and Hosmer-Lemeshow test for calibration. The regression task was evaluated by comparing median actual and predicted discharge, loss penalty function (LPF) ((actual-predicted)/actual) and calculating root mean squared relative errors (RMSRE).ResultsMedian (P25-P75) ICU length of stay was 3 (2-5) days. For classification, the GP model showed an aROC of 0.758 which was significantly higher than the predictions by nurses, but not better than EuroSCORE and physicians. The GP had the best calibration, with a Brier Score of 0.179 and Hosmer-Lemeshow p-value of 0.382. For regression, GP had the highest proportion of patients with a correctly predicted day of discharge (40%), which was significantly better than the EuroSCORE (p < 0.001) and nurses (p = 0.044) but equivalent to physicians. GP had the lowest RMSRE (0.408) of all predictive models.ConclusionsA GP model that uses PDMS data of the first 4 hours after admission in the ICU of scheduled adult cardiac surgery patients was able to predict discharge from the ICU as a classification as well as a regression task. The GP model demonstrated a significantly better discriminative power than the EuroSCORE and the ICU nurses, and at least as good as predictions done by ICU physicians. The GP model was the only well calibrated model.

[1]  J. Ramon,et al.  Machine learning techniques to examine large patient databases. , 2009, Best practice & research. Clinical anaesthesiology.

[2]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[3]  J. Zimmerman,et al.  Intensive care unit length of stay: Benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV* , 2006, Critical care medicine.

[4]  Marco Ranucci,et al.  An adjusted EuroSCORE model for high-risk cardiac patients. , 2009, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[5]  T. Alonzo Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating By Ewout W. Steyerberg , 2009 .

[6]  Laura P Coombs,et al.  The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models. , 2003, The Annals of thoracic surgery.

[7]  Tudor Toma,et al.  Discovery and inclusion of SOFA score episodes in mortality prediction , 2007, J. Biomed. Informatics.

[8]  J. Zimmerman,et al.  Predicting Outcomes for Cardiac Surgery Patients After Intensive Care Unit Admission , 2008, Seminars in cardiothoracic and vascular anesthesia.

[9]  S. Nashef,et al.  Eurp-pean system for cardiac operative risk evaluation (Euro SCORE) , 1999 .

[10]  B. Keogh,et al.  Fifth National Adult Cardiac Surgical Database Report 2003: Improving outcomes for patients , 2004 .

[11]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[12]  A. H. Murphy A New Vector Partition of the Probability Score , 1973 .

[13]  J. Wennberg,et al.  Multivariate Prediction of In‐Hospital Mortality Associated With Coronary Artery Bypass Graft Surgery , 1992 .

[14]  A. Bernstein,et al.  A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. , 1989, Circulation.

[15]  Maurice Bruynooghe,et al.  Mining data from intensive care patients , 2007, Adv. Eng. Informatics.

[16]  Geert Meyfroidt,et al.  How to implement information technology in the operating room and the intensive care unit. , 2009, Best practice & research. Clinical anaesthesiology.

[17]  G. Beck,et al.  Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients. A clinical severity score. , 1992, JAMA.

[18]  Ewout W. Steyerberg,et al.  Validation of Prediction Models , 2019, Statistics for Biology and Health.

[19]  D. Cheng,et al.  Risk Stratification Models for Cardiac Surgery , 2008, Seminars in cardiothoracic and vascular anesthesia.

[20]  A. Nierich,et al.  Prediction Models for Prolonged Intensive Care Unit Stay After Cardiac Surgery: Systematic Review and Validation Study , 2010, Circulation.

[21]  F. Biancari,et al.  Is it possible to improve the accuracy of EuroSCORE? , 2009, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[22]  J. Zimmerman,et al.  Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients* , 2006, Critical care medicine.

[23]  Ewout W. Steyerberg,et al.  Overfitting and optimism in prediction models , 2009 .

[24]  Pantpis Sakornpant,et al.  First National Adult Cardiac Surgical Database Report , 2011 .

[25]  Salvatore Colangelo,et al.  Can EuroSCORE predict direct costs of cardiac surgery? , 2003, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[26]  S. Lemeshow,et al.  European system for cardiac operative risk evaluation (EuroSCORE). , 1999, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[27]  W. Knaus,et al.  The use of APACHE III to evaluate ICU length of stay, resource use, and mortality after coronary artery by-pass surgery. , 1995, The Journal of cardiovascular surgery.

[28]  R. Manjoney,et al.  Clinical information systems market - an insider's view. , 2004, Journal of critical care.

[29]  G. Brier VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[30]  M. Hippeläinen,et al.  Intra-institutional prediction of outcome after cardiac surgery: comparison between a locally derived model and the EuroSCORE. , 2000, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[31]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[32]  L. Bossaert,et al.  Is EuroSCORE useful in the prediction of extended intensive care unit stay after cardiac surgery? , 2009, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[33]  水谷 博之,et al.  SVM (Support Vector Machine) , 1999 .

[34]  S M Pincus,et al.  Approximate entropy as a measure of system complexity. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Carsten Lührs,et al.  EuroSCORE predicts intensive care unit stay and costs of open heart surgery. , 2004, The Annals of thoracic surgery.

[36]  I. Toumpoulis,et al.  Does EuroSCORE predict length of stay and specific postoperative complications after cardiac surgery? , 2005, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[37]  E. Steyerberg Clinical Prediction Models , 2008, Statistics for Biology and Health.