Assessment of six mortality prediction models in patients admitted with severe sepsis and septic shock to the intensive care unit: a prospective cohort study

IntroductionWe conducted the present study to assess the validity of mortality prediction systems in patients admitted to the intensive care unit (ICU) with severe sepsis and septic shock. We included Acute Physiology and Health Evaluation (APACHE) II, Simplified Acute Physiology Score (SAPS) II, Mortality Probability Model (MPM) II0 and MPM II24 in our evaluation. In addition, SAPS II and MPM II24 were customized for septic patients in a previous study, and the customized versions were included in this evaluation.Materials and methodThis cohort, prospective, observational study was conducted in a tertiary care medical/surgical ICU. Consecutive patients meeting the diagnostic criteria for severe sepsis and septic shock during the first 24 hours of ICU admission between March 1999 and August 2001 were included. The data necessary for mortality prediction were collected prospectively as part of the ongoing ICU database. Predicted and actual mortality rates, and standardized mortality ratio were calculated. Calibration was assessed using Lemeshow–Hosmer goodness of fit C-statistic. Discrimination was assessed using receiver operating characteristic curves.ResultsThe overall mortality prediction was adequate for all six systems because none of the standardized mortality ratios differed significantly from 1. Calibration was inadequate for APACHE II, SAPS II, MPM II0 and MPM II24. However, the customized version of SAPS II exhibited significantly improved calibration (C-statistic for SAPS II 23.6 [P = 0.003] and for customized SAPS II 11.5 [P = 0.18]). Discrimination was best for customized MPM II24 (area under the receiver operating characteristic curve 0.826), followed by MPM II24 and customized SAPS II.ConclusionAlthough general ICU mortality system models had accurate overall mortality prediction, they had poor calibration. Customization of SAPS II and, to a lesser extent, MPM II24 improved calibration. The customized model may be a useful tool when evaluating outcomes in patients with sepsis.

[1]  E. Draper,et al.  APACHE II: A severity of disease classification system , 1985, Critical care medicine.

[2]  D. Edbrooke,et al.  The patient-related costs of care for sepsis patients in a United Kingdom adult general intensive care unit. , 1999, Critical care medicine.

[3]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[4]  D. Hosmer,et al.  A review of goodness of fit statistics for use in the development of logistic regression models. , 1982, American journal of epidemiology.

[5]  M. S. Rangel-Frausto,et al.  The epidemiology of bacterial sepsis. , 1999, Infectious disease clinics of North America.

[6]  J. E. Carceller.  American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference: Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis , 1992, Critical care medicine.

[7]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[8]  M. Langer,et al.  The Italian SEPSIS study: Preliminary results on the incidence and evolution of SIRS, sepsis, severe sepsis and septic shock , 1995, Intensive Care Medicine.

[9]  F Doyon,et al.  Incidence, risk factors, and outcome of severe sepsis and septic shock in adults. A multicenter prospective study in intensive care units. French ICU Group for Severe Sepsis. , 1995, JAMA.

[10]  G. Clermont,et al.  Epidemiology of severe sepsis in the United States: Analysis of incidence, outcome, and associated costs of care , 2001, Critical care medicine.

[11]  Derek C. Angus,et al.  Epidemiology of sepsis: An update , 2001, Critical care medicine.

[12]  Stanley Lemeshow,et al.  Modeling the Severity of Illness of ICU Patients , 2001 .

[13]  S Lemeshow,et al.  Customized probability models for early severe sepsis in adult intensive care patients. Intensive Care Unit Scoring Group. , 1995, JAMA.

[14]  D M Linton,et al.  Severity of disease classification system. , 1991, Critical care medicine.

[15]  J. Vincent,et al.  Has the mortality of septic shock changed with time. , 1998, Critical care medicine.

[16]  S. Lemeshow,et al.  Modeling the severity of illness of ICU patients. A systems update. , 1994, JAMA.

[17]  M. Levy,et al.  2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference , 2003, Intensive care medicine.

[18]  H. S. Warren,et al.  Risks and benefits of activated protein C treatment for severe sepsis. , 2002, The New England journal of medicine.

[19]  W. Knaus,et al.  Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. , 1992, Chest.

[20]  S. Lowry,et al.  An overview of mortality risk prediction in sepsis. , 1995, Critical care medicine.

[21]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. , 1993, JAMA.

[22]  Ziad A Memish,et al.  Mecca bound: the challenges ahead. , 2006, Journal of travel medicine.

[23]  G. Clermont,et al.  Short‐term and long‐term outcome prediction with the Acute Physiology and Chronic Health Evaluation II system after orthotopic liver transplantation , 2000, Critical care medicine.

[24]  Janelle Klar,et al.  Customized probability models for early severe sepsis in adult intensive care patients. Intensive Care Unit Scoring Group. , 1995, JAMA.

[25]  J Ean,et al.  Efficacy and safety of recombinant human activated protein C for severe sepsis. , 2001, The New England journal of medicine.

[26]  Mitchell M. Levy,et al.  2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference , 2003, Intensive Care Medicine.

[27]  François Gouin,et al.  Incidence, Risk Factors, and Outcome of Severe Sepsis and Septic Shock in Adults: A Multicenter Prospective Study in Intensive Care Units , 1995 .

[28]  J. Helterbrand,et al.  Efficacy and safety of recombinant human activated protein C for severe sepsis , 2003 .

[29]  D. Goldhill,et al.  Outcome of intensive care patients in a group of British intensive care units. , 1998, Critical care medicine.

[30]  M. Suistomaa,et al.  Customised prediction models based on APACHE II and SAPS II scores in patients with prolonged length of stay in the ICU , 2002, Intensive Care Medicine.

[31]  J. le Gall,et al.  Modeling the severity of illness of ICU patients. , 2001, European journal of internal medicine.

[32]  S. Lemeshow,et al.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. , 1993, JAMA.