Performance of the EuroSCORE Models in Emergency Cardiac Surgery

Background—Accurate risk-adjustment models are useful for clinical decision making and are important for minimizing any tendency toward risk-averse clinical practice. In cardiac surgery, emergency patients are potentially at greatest risk of inappropriate risk-averse clinical decisions. UK cardiac surgery outcomes are currently risk-adjusted with EuroSCORE models. The objective of this study was to assess the performance of the EuroSCORE models in emergency cardiac surgery. Methods and Results—The National Institute for Cardiovascular Outcomes Research database was used to identify adult cardiac surgery procedures performed in the United Kingdom between April 2008 and March 2011. A subset of procedures (July 2010–March 2011) was used for EuroSCORE II validation. The outcome measure was in-hospital mortality. Model calibration (Hosmer–Lemeshow test, calibration plots, calculation of calibration intercept and slope) and discrimination (area under receiver-operating characteristic curve [area under the curve]) were assessed. In total, 109 988 cardiac procedures at 41 hospitals were included, of which 3342 were defined as emergency procedures. Compared with performance in all cardiac surgery and nonemergency cardiac surgery, the logistic EuroSCORE and EuroSCORE II models had poorer discrimination (area under the curve, 0.703 and 0.690, respectively) and poorer calibration for emergency surgery. The EuroSCORE risk factors of female sex, chronic pulmonary disease, neurological disease, active endocarditis, unstable angina, recent myocardial infarction, and pulmonary hypertension were not identified as important risk factors for emergency cardiac surgery. Conclusions—Both EuroSCORE models demonstrated poor calibration and comparatively poor discrimination for emergency cardiac surgery. This has important implications when these models are used for clinical decision making or to adjust governance analyses.

[1]  E J Topol,et al.  Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes. , 1996, Circulation.

[2]  S. Nashef,et al.  Outcomes in emergency redo cardiac surgery: cost, benefit and risk assessment. , 2003, Interactive cardiovascular and thoracic surgery.

[3]  F. Edwards,et al.  The Society of Thoracic Surgeons 2008 cardiac surgery risk models: introduction. , 2009, The Annals of thoracic surgery.

[4]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.

[5]  Patrick S Romano,et al.  Size matters to a model's fit. , 2007, Critical care medicine.

[6]  Marco Ranucci,et al.  An adjusted EuroSCORE model for high-risk cardiac patients. , 2009, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[7]  B. Bridgewater,et al.  The logistic EuroSCORE in cardiac surgery: how well does it predict operative risk? , 2006, Heart.

[8]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[9]  A. Bryan,et al.  How does EuroSCORE II perform in UK cardiac surgery; an analysis of 23 740 patients from the Society for Cardiothoracic Surgery in Great Britain and Ireland National Database , 2012, Heart.

[10]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[11]  F. Biancari,et al.  Is it possible to improve the accuracy of EuroSCORE? , 2009, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[12]  P. Pinho,et al.  Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons. , 2000, Revista portuguesa de cardiologia : orgao oficial da Sociedade Portuguesa de Cardiologia = Portuguese journal of cardiology : an official journal of the Portuguese Society of Cardiology.

[13]  Adult open heart surgery in New York State. An analysis of risk factors and hospital mortality rates. , 1991, JAMA.

[14]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[15]  T. Carrel,et al.  Risk-stratification in thoracic aortic surgery: should the EuroSCORE be modified? , 2004, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[16]  S. Nashef,et al.  The logistic EuroSCORE , 2003 .

[17]  T. Ferris,et al.  Public release of clinical outcomes data—online CABG report cards. , 2010, The New England journal of medicine.

[18]  N. Cook Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.

[19]  A. Agresti,et al.  Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions , 1998 .

[20]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[21]  S. Nashef,et al.  Logistic or additive EuroSCORE for high-risk patients? , 2003, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[22]  S. Nashef,et al.  The EuroSCORE risk stratification system in the current era: how accurate is it and what should be done if it is inaccurate? , 2009, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[23]  A M Epstein,et al.  Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. , 1996, The New England journal of medicine.

[24]  B. Bridgewater,et al.  Does the choice of risk-adjustment model influence the outcome of surgeon-specific mortality analysis? A retrospective analysis of 14 637 patients under 31 surgeons , 2007, Heart.

[25]  D. Rosborough,et al.  Nonelective cardiac surgery in the elderly: is it justified? , 2010, The Journal of thoracic and cardiovascular surgery.

[26]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[27]  S. Lemeshow,et al.  European system for cardiac operative risk evaluation (EuroSCORE). , 1999, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[28]  M. Kamler,et al.  Society of Thoracic Surgeons score is superior to the EuroSCORE determining mortality in high risk patients undergoing isolated aortic valve replacement. , 2009, The Annals of thoracic surgery.

[29]  E. Steyerberg,et al.  [Regression modeling strategies]. , 2011, Revista espanola de cardiologia.

[30]  B. Bridgewater,et al.  Demonstrating quality: The sixth National Adult Cardiac Surgery database report: , 2009 .

[31]  M. Radford,et al.  Impact of the New York State Cardiac Surgery and Percutaneous Coronary Intervention Reporting System on the management of patients with acute myocardial infarction complicated by cardiogenic shock. , 2008, American heart journal.

[32]  R. D. Alley,et al.  The society of thoracic surgeons. , 1976, The Annals of thoracic surgery.

[33]  Samer A M Nashef,et al.  EuroSCORE II. , 2012, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[34]  Frank E. Harrell,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[35]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[36]  H. Akaike A new look at the statistical model identification , 1974 .

[37]  J. Zimmerman,et al.  Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited* , 2007, Critical care medicine.

[38]  Rolf H H Groenwold,et al.  Performance of the original EuroSCORE. , 2012, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[39]  B. Bridgewater Cardiac registers: the adult cardiac surgery register , 2010, Heart.