Predictive mortality models are not like fine wine

The authors of a recent paper have described an updated simplified acute physiology score (SAPS) II mortality model developed on patient data from 1998 to 1999. Hospital mortality models have a limited range of applicability. SAPS II, Acute Physiology, Age, and Chronic Health Evaluation (APACHE) III, and mortality probability model (MPM)-II, which were developed in the early 1990s, have shown a decline in predictive accuracy as the models age. The deterioration in accuracy is manifested by a decline in the models' calibration. In particular, mortality tends to get over predicted when older models are applied to more contemporary data, which in turn leads to 'grade inflation' when benchmarking intensive care unit (ICU) performance. Although the authors claim that their updated SAPS II can be used for benchmarking ICU performance, it seems likely that this model might already be out of calibration for patient data collected in 2005 and beyond. Thus, the updated SAPS II model may be interesting for historical purposes, but it is doubtful that it can be an accurate tool for benchmarking data from contemporary populations.

[1]  W. Knaus,et al.  The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. , 1991, Chest.

[2]  E. Draper,et al.  A multicenter description of intermediate-care patients: comparison with ICU low-risk monitor patients. , 2002, Chest.

[3]  J. Norrie,et al.  Assessment of the performance of five intensive care scoring models within a large Scottish database , 2000, Critical care medicine.

[4]  François Hemery,et al.  Mortality prediction using SAPS II: an update for French intensive care units , 2005, Critical care.

[5]  S. Lemeshow,et al.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. , 1993, JAMA.

[6]  Marc J Popovich If most intensive care units are graduating with honors, is it genuine quality or grade inflation? , 2002, Critical care medicine.

[7]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. , 1993, JAMA.

[8]  T. Osler,et al.  Rating the quality of intensive care units: Is it a function of the intensive care unit scoring system?* , 2002, Critical care medicine.

[9]  T. Higgins,et al.  SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description. , 2005 .

[10]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study , 1993 .

[11]  James Deddens,et al.  Variation in outcomes in Veterans Affairs intensive care units with a computerized severity measure* , 2005, Critical care medicine.