A revised method to assess intensive care unit clinical performance and resource utilization*

Objective:In 1994, Rapoport et al. published a two-dimensional graphical tool for benchmarking intensive care units (ICUs) using a Mortality Probability Model (MPM0-II) to assess clinical performance and a Weighted Hospital Days scale (WHD-94) to assess resource utilization. MPM0-II and WHD-94 do not calibrate on contemporary data, giving users of the graph an inflated assessment of their ICU’s performance. MPM0-II was recently updated (MPM0-III) but not the model for predicting resource utilization. The objective was to develop a new WHD model and revised Rapoport-Teres graph. Design:Multicenter cohort study. Setting:One hundred thirty-five ICUs in 98 hospitals participating in Project IMPACT. Patients:Patients were 124,855 MPM0-II eligible Project IMPACT patients treated between March 2001 and June 2004. Interventions:None. Measurements and Main Results:WHD was redefined as 4 units for the first day of each ICU stay, 2.5 units for each additional ICU day, and 1 unit for each non-ICU day after the first ICU discharge. Stepwise linear regression was used to construct a model to predict ICU-specific log average WHD from 39 candidate variables available in Project IMPACT. The updated WHD model has four independent variables: percent of patients dying in the hospital, percent of unscheduled surgical patients, percent of patients on mechanical ventilation within 1 hr of ICU admission, and percent discharged from the ICU to an external post-acute care facility. The first three variables increase average WHD and the last decreases it. The new model has good performance (R2 = 0.47) and, when combined with MPM0-II, provides a well-calibrated Rapoport-Teres graph. Conclusions:A new WHD model has been derived from a large, contemporary critical care database and, when used with MPM0-III, updates a popular method for benchmarking ICUs. Project IMPACT participants will likely perceive a decline in their ICU performance coordinates due to the recalibrated graph and should instead focus on their unit’s performance relative to their peers.

[1]  S. Lemeshow,et al.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. , 1993, JAMA.

[2]  Marc J Popovich.  If most intensive care units are graduating with honors, is it genuine quality or grade inflation? , 2002, Critical care medicine.

[3]  C. Piech,et al.  Daily cost of an intensive care unit day: The contribution of mechanical ventilation* , 2005, Critical care medicine.

[4]  G. Iapichino,et al.  Variable costs of ICU patients: a multicenter prospective study , 2006, Intensive Care Medicine.

[5]  A. Kramer,et al.  Predictive mortality models are not like fine wine , 2005, Critical care.

[6]  Yaseen Arabi,et al.  A prospective study of prolonged stay in the intensive care unit: predictors and impact on resource utilization. , 2002, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[7]  L. Ryan,et al.  Concurrent prediction of hospital mortality and length of stay from risk factors on admission. , 2002, Health services research.

[8]  F B Rogers,et al.  Predicting survival, length of stay, and cost in the surgical intensive care unit: APACHE II versus ICISS. , 1998, The Journal of trauma.

[9]  J. Zimmerman,et al.  The use of benchmarking to identify top performing critical care units: a preliminary assessment of their policies and practices. , 2003, Journal of critical care.

[10]  T. Ganz,et al.  Using a collaborative weaning plan to decrease duration of mechanical ventilation and length of stay in the intensive care unit for patients receiving long-term ventilation. , 2002, American journal of critical care : an official publication, American Association of Critical-Care Nurses.

[11]  James R Carpenter,et al.  Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom* , 2006, Critical care medicine.

[12]  R. Melotti,et al.  Effectiveness and efficiency of intensive care medicine: variable costs in different diagnosis groups , 2004, Acta anaesthesiologica Scandinavica.

[13]  T. Breurch,et al.  A simple test for heteroscedasticity and random coefficient variation (econometrica vol 47 , 1979 .

[14]  S Lemeshow,et al.  A method for assessing the clinical performance and cost‐effectiveness of intensive care units: A multicenter inception cohort study , 1994, Critical care medicine.

[15]  J. Lave,et al.  The effect of managed care on ICU length of stay: implications for medicare. , 1996, JAMA.

[16]  Carnell Cooper,et al.  Daily multidisciplinary rounds shorten length of stay for trauma patients. , 2003, The Journal of trauma.

[17]  J. Zimmerman,et al.  Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients* , 2006, Critical care medicine.

[18]  A. Sear,et al.  Measuring hospital efficiency: a comparison of two approaches. , 2000, Health services research.

[19]  P. Marik,et al.  What’s in a day? Determining intensive care unit length of stay , 2000, Critical care medicine.

[20]  Todd Dorman,et al.  Predicting patient outcomes, futility, and resource utilization in the intensive care unit: the role of severity scoring systems and general outcome prediction models. , 2005, Mayo Clinic proceedings.

[21]  Roger Resar,et al.  Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. , 2005, Joint Commission journal on quality and patient safety.

[22]  S. Cook,et al.  Project IMPACT: Results from a Pilot Validity Study of a New Observational Database , 2002, Critical care medicine.

[23]  A. Shorr,et al.  An update on cost-effectiveness analysis in critical care , 2002, Current opinion in critical care.

[24]  S. Lemeshow,et al.  Length of Stay Data as a Guide to Hospital Economic Performance for ICU Patients , 2003, Medical care.

[25]  D. Teres,et al.  Assessing contemporary intensive care unit outcome: An updated Mortality Probability Admission Model (MPM0-III)* , 2007, Critical care medicine.

[26]  Laura P Coombs,et al.  The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models. , 2003, The Annals of thoracic surgery.

[27]  Robert Tibshirani,et al.  Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[28]  L. Hydo,et al.  Utility of illness severity scoring for prediction of prolonged surgical critical care. , 1996, The Journal of trauma.

[29]  Peter J Pronovost,et al.  Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. , 2002, JAMA.

[30]  Stanley Lemeshow,et al.  Early indicators of prolonged intensive care unit stay: Impact of illness severity, physician staffing, and pre–intensive care unit length of stay , 2003, Critical care medicine.

[31]  Malik Beshir Malik,et al.  Applied Linear Regression , 2005, Technometrics.

[32]  U. Ruttimann,et al.  PRISM III: an updated Pediatric Risk of Mortality score. , 1996, Critical care medicine.

[33]  Peter J Pronovost,et al.  Intensive care unit physician staffing: Financial modeling of the Leapfrog standard* , 2004, Critical care medicine.

[34]  D. Pregibon.  Goodness of Link Tests for Generalized Linear Models , 1980 .

[35]  T. Osler,et al.  Rating the quality of intensive care units: Is it a function of the intensive care unit scoring system?* , 2002, Critical care medicine.

[36]  B. Afessa,et al.  Intensivist-to-bed ratio: association with outcomes in the medical ICU. , 2005, Chest.

[37]  J. S. Long,et al.  Some Specification Tests for the Linear Regression Model , 1992 .