Evaluation of acute physiology and chronic health evaluation III predictions of hospital mortality in an independent database.

OBJECTIVE To assess the accuracy and validity of Acute Physiology and Chronic Health Evaluation (APACHE) III hospital mortality predictions in an independent sample of U.S. intensive care unit (ICU) admissions. DESIGN Nonrandomized, observational, cohort study. SETTING Two hundred eighty-five ICUs in 161 U.S. hospitals, including 65 members of the Council of Teaching Hospitals and 64 nonteaching hospitals. PATIENTS A consecutive sample of 37,668 ICU admissions during 1993 to 1996; including 25,448 admissions at hospitals with >400 beds and 1,074 admissions at hospitals with <200 beds. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS We used demographic, clinical, and physiologic information recorded during ICU day 1 and the APACHE III equation to predict the probability of hospital mortality for each patient. We compared observed and predicted mortality for all admissions and across patient subgroups and assessed predictive accuracy using tests of discrimination and calibration. Aggregate hospital death rate was 12.35% and predicted hospital death rate was 12.27% (p =.541). The model discriminated between survivors and nonsurvivors well (area under receiver operating curve = 0.89). A calibration curve showed that the observed number of hospital deaths was close to the number of deaths predicted by the model, but when tested across deciles of risk, goodness-of-fit (Hosmer-Lemeshow statistic, chi-square = 48.71, 8 degrees of freedom, p< .0001) was not perfect. Observed and predicted hospital mortality rates were not significantly (p < .01) different for 55 (84.6%) of APACHE III's 65 specific ICU admission diagnoses and for 11 (84.6%) of the 13 residual organ system-related categories. The most frequent diagnoses with significant (p < .01) differences between observed and predicted hospital mortality rates included acute myocardial infarction, drug overdose, nonoperative head trauma, and nonoperative multiple trauma. CONCLUSIONS APACHE III accurately predicted aggregate hospital mortality in an independent sample of U.S. ICU admissions. Further improvements in calibration can be achieved by more precise disease labeling, improved acquisition and weighting of neurologic abnormalities, adjustments that reflect changes in treatment outcomes over time, and a larger national database.

[1]  M Pine,et al.  Predictions of Hospital Mortality Rates: A Comparison of Data Sources , 1997, Annals of Internal Medicine.

[2]  A. Laupacis,et al.  Clinical prediction rules. A review and suggested modifications of methodological standards. , 1997, JAMA.

[3]  R. Moreno,et al.  Outcome prediction in intensive care: results of a prospective, multicentre, Portuguese study , 1997, Intensive Care Medicine.

[4]  B. Taylor,et al.  Prediction of outcome from intensive care: a prospective cohort study comparing Acute Physiology and Chronic Health Evaluation II and III prognostic systems in a United Kingdom intensive care unit. , 1997, Critical care medicine.

[5]  J. Lipman,et al.  Poor discriminatory performance of the Pediatric Risk of Mortality (PRISM) score in a South African intensive care unit. , 1996, Critical care medicine.

[6]  J. Sullivan,et al.  Comparative assessment of pediatric intensive care in Moscow, the Russian Federation: a prospective, multicenter study. , 1996, Critical care medicine.

[7]  A M Epstein,et al.  Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. , 1996, The New England journal of medicine.

[8]  W. Knaus,et al.  Intensive care unit admissions with cirrhosis: Risk‐stratifying patient groups and predicting individual survival , 1996, Hepatology.

[9]  J H Wasson,et al.  Clinical Prediction Rules: Have They Come of Age? , 1996 .

[10]  S Lemeshow,et al.  Factors affecting the performance of the models in the Mortality Probability Model II system and strategies of customization: a simulation study. , 1996, Critical care medicine.

[11]  A R Localio,et al.  A Report Card for Report Cards , 1995, Annals of Internal Medicine.

[12]  J. Bion,et al.  A comparison of severity of illness scoring systems for intensive care unit patients: results of a multicenter, multinational study. The European/North American Severity Study Group. , 1995, Critical care medicine.

[13]  Frank E. Harrell,et al.  Identification of Comatose Patients at High Risk for Death or Severe Disability , 1995 .

[14]  N. Wintfeld,et al.  Report cards on cardiac surgeons. Assessing New York State's approach. , 1995, The New England journal of medicine.

[15]  G. Guyatt,et al.  Determinants in Canadian health care workers of the decision to withdraw life support from the critically ill. Canadian Critical Care Trials Group. , 1995, JAMA.

[16]  J. Ghajar,et al.  Survey of critical care management of comatose, head-injured patients in the United States. , 1995, Critical care medicine.

[17]  W. Knaus,et al.  The use of APACHE III to evaluate ICU length of stay, resource use, and mortality after coronary artery by-pass surgery. , 1995, The Journal of cardiovascular surgery.

[18]  P. Mayo,et al.  Outcome of intensive care in patients with HIV infection. , 1995, Chest.

[19]  R. Wachter,et al.  Cost and outcome of intensive care for patients with AIDS, Pneumocystis carinii pneumonia, and severe respiratory failure. , 1995, JAMA.

[20]  W. Knaus,et al.  DAILY OUTCOME ESTIMATES FOR PATIENTS WITH ADVANCED MALIGNANCY AND RESPIRATORY FAILURE , 1995 .

[21]  S. Lemeshow,et al.  Modeling the severity of illness of ICU patients. A systems update. , 1994, JAMA.

[22]  D. Wagner,et al.  Daily prognostic estimates for critically ill adults in intensive care units: Results from a prospective, multicenter, inception cohort analysis , 1994, Critical care medicine.

[23]  J H Kerr,et al.  Intensive Care Society's Acute Physiology and Chronic Health Evaluation (APACHE II) study in Britain and Ireland: A prospective, multicenter, cohort study comparing two methods for predicting outcome for adult intensive care patients , 1994, Critical care medicine.

[24]  N. Binkin,et al.  Differential survival of patients with AIDS according to the 1987 and 1993 CDC case definitions. , 1994, JAMA.

[25]  R. Chesnut,et al.  Early, routine paralysis for intracranial pressure control in severe head injury: Is it necessary? , 1993, Critical care medicine.

[26]  R. Califf,et al.  Scorecard Cardiovascular Medicine: Its Impact and Future Directions , 1994, Annals of Internal Medicine.

[27]  M. A. Kelley,et al.  Errors and bias in using predictive scoring systems. , 1994, Critical care clinics.

[28]  U. Ruttimann,et al.  Statistical approaches to development and validation of predictive instruments. , 1994, Critical care clinics.

[29]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study , 1993 .

[30]  S. Lemeshow,et al.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. , 1993, JAMA.

[31]  J H Kerr,et al.  Intensive Care Society's APACHE II study in Britain and Ireland--I: Variations in case mix of adult admissions to general intensive care units and impact on outcome. , 1993, BMJ.

[32]  J H Kerr,et al.  Intensive Care Society's APACHE II study in Britain and Ireland--II: Outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. , 1993, BMJ.

[33]  E. DeLong,et al.  Discordance of Databases Designed for Claims Payment versus Clinical Information Systems: Implications for Outcomes Research , 1993, Annals of Internal Medicine.

[34]  D. Wagner,et al.  Glasgow Coma Scale score in the evaluation of outcome in the intensive care unit: Findings from the Acute Physiology and Chronic Health Evaluation III study , 1993, Critical care medicine.

[35]  W. Knaus,et al.  Variations in Mortality and Length of Stay in Intensive Care Units , 1993, Annals of Internal Medicine.

[36]  A. Bersten,et al.  Prospective evaluation of residents and nurses as severity score data collectors , 1992, Critical care medicine.

[37]  D. Schuster.  Predicting outcome after ICU admission. The art and science of assessing risk. , 1992, Chest.

[38]  W. Knaus,et al.  Reliability of a measure of severity of illness: acute physiology of chronic health evaluation--II. , 1992, Journal of clinical epidemiology.

[39]  W. Knaus,et al.  The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. , 1991, Chest.

[40]  G. Blackburn,et al.  Elective intrahospital admissions versus acute interhospital transfers to a surgical intensive care unit: cost and outcome prediction. , 1991, The Journal of trauma.

[41]  G. Blackburn,et al.  Surgical intensive care unit resource use in a specialty referral hospital: I. Predictors of early death and cost implications. , 1991, Surgery.

[42]  R H Brook,et al.  Explaining variations in hospital death rates. Randomness, severity of illness, quality of care. , 1990, JAMA.

[43]  N. Joshua,et al.  Multidisciplinary intensive care unit at Cecilia Makiwane Hospital, Mdantsane, Ciskei. , 1989, South African medical journal = Suid-Afrikaanse tydskrif vir geneeskunde.

[44]  D Draper,et al.  Interpreting hospital mortality data. The role of clinical risk adjustment. , 1988, JAMA.

[45]  R Simon,et al.  Why predictive indexes perform less well in validation studies. Is it magic or methods? , 1987, Archives of internal medicine.

[46]  H. Luft,et al.  Evaluating individual hospital quality through outcome statistics. , 1986, JAMA.

[47]  Michael J. Strauss,et al.  Rationing of intensive care unit services. An everyday occurrence. , 1986, JAMA.

[48]  H. Sox,et al.  Clinical prediction rules. Applications and methodological standards. , 1985, The New England journal of medicine.

[49]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[50]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[51]  D. Hosmer,et al.  A review of goodness of fit statistics for use in the development of logistic regression models. , 1982, American journal of epidemiology.