Monitoring the evolutionary process of quality: risk-adjusted charting to track outcomes in intensive care.

OBJECTIVE To present graphical procedures for prospectively monitoring outcomes in the intensive care unit. DESIGN Observational study: risk-adjusted control chart analysis of a case series. SETTING Tertiary referral adult intensive care unit: Princess Alexandra Hospital, Brisbane, Australia. PATIENTS A total of 3398 intensive care unit admissions from January 1, 1995, to January 1, 1998. CONCLUSIONS Risk-adjusted process control charting procedures for continuous monitoring of intensive care unit outcomes are proposed as quality management tools. A modified Shewhart p chart and cumulative sum process control chart, using the Acute Physiology and Chronic Health Evaluation III model mortality prediction for risk adjustment, are presented. The risk-adjusted p chart summarizes performance at arbitrary intervals and plots observed against predicted mortality rate to detect large changes in risk-adjusted mortality. The risk-adjusted cumulative sum procedure is a likelihood-based scoring method that adjusts for estimated risk of death, accumulating evidence from outcomes of all previous patients. It formally tests the hypothesis of a change in the odds of death. In this application, we detected a decrease from above to predicted risk-adjusted mortality. This was temporally related to increased senior staffing levels and enhanced ongoing multidisciplinary review of practice, quality improvement, and educational activities. Formulas and analyses are provided as appendices.

[1]  D. Cook,et al.  Performance of APACHE III models in an Australian ICU. , 2000, Chest.

[2]  A F Mackintosh,et al.  Use of cumulative mortality data in patients with acute myocardial infarction for early detection of variation in clinical practice: observational study , 2001, BMJ : British Medical Journal.

[3]  A J Rotondi,et al.  Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland Health Quality Choice. , 1999, Chest.

[4]  R J Cook,et al.  Monitoring paired binary surgical outcomes using cumulative sum charts. , 1999, Statistics in medicine.

[5]  S. Williams,et al.  ERCP: a review of technical competency and workload in a small unit. , 1997, Gastrointestinal endoscopy.

[6]  S. Steiner,et al.  Monitoring surgical performance using risk-adjusted cumulative sum charts. , 2000, Biostatistics.

[7]  B R Parry,et al.  Competency and the colonoscopist: a learning curve. , 1991, The Australian and New Zealand journal of surgery.

[8]  G. Apolone,et al.  The performance of SAPS II in a cohort of patients admitted to 99 Italian ICUs: Results from GiViTI , 1996, Intensive Care Medicine.

[9]  R. Carey,et al.  Measuring for improvement: from Toyota to thoracic surgery. , 1999, Annals of Thoracic Surgery.

[10]  Peter Pronovost,et al.  Reorganizing the delivery of intensive care could improve efficiency and save lives. , 2002, Journal of evaluation in clinical practice.

[11]  P. Landais,et al.  Evaluation of severity scoring systems in ICUs—translation, conversion and definition ambiguities as a source of inter-observer variability in Apache II, SAPS and OSF , 1995, Intensive Care Medicine.

[12]  W. Knaus,et al.  The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. , 1991, Chest.

[13]  S Gallivan,et al.  Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice? , 2000, Heart.

[14]  S Gallivan,et al.  Detection of changes in mortality after heart surgery , 1998, BMJ.

[15]  G. Marshall,et al.  Time series monitors of outcomes. A new dimension for measuring quality of care. , 1998, Medical care.

[16]  J. L. Gall,et al.  APACHE II--a severity of disease classification system. , 1986, Critical care medicine.

[17]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. , 1993, JAMA.

[18]  S M Williams,et al.  Quality control: an application of the cusum. , 1992, BMJ.

[19]  J C Benneyan,et al.  Use and interpretation of statistical quality control charts. , 1998, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[20]  Chris Sherlaw-Johnson,et al.  Monitoring the results of cardiac surgery by variable life-adjusted display , 1997, The Lancet.

[21]  P Littlejohns,et al.  Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery , 1998, BMJ.

[22]  W. Knaus,et al.  Interhospital comparisons of patient outcome from intensive care: importance of lead-time bias. , 1989, Critical care medicine.

[23]  Janelle Klar,et al.  Customized probability models for early severe sepsis in adult intensive care patients. Intensive Care Unit Scoring Group. , 1995, JAMA.

[24]  Stephen A. Freitas,et al.  Modern Industrial Statistics: Design and Control of Quality and Reliability , 1999, Technometrics.

[25]  S Lemeshow,et al.  Customized probability models for early severe sepsis in adult intensive care patients. Intensive Care Unit Scoring Group. , 1995, JAMA.

[26]  R J Cook,et al.  Risk-Adjusted Monitoring of Binary Surgical Outcomes , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[27]  J D Flora,et al.  A method for comparing survival of burn patients to a standard survival curve. , 1978, The Journal of trauma.

[28]  W J Sibbald,et al.  Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. , 1999, Critical care medicine.

[29]  S Lemeshow,et al.  Mortality probability models for patients in the intensive care unit for 48 or 72 hours: A prospective, multicenter study , 1994, Critical care medicine.

[30]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study , 1993 .

[31]  S Lemeshow,et al.  Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. , 1996, Critical care medicine.

[32]  S. Lemeshow,et al.  Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. , 1993, JAMA.

[33]  S Lemeshow,et al.  Factors affecting the performance of the models in the Mortality Probability Model II system and strategies of customization: a simulation study. , 1996, Critical care medicine.

[34]  M R de Leval,et al.  Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switch operations. , 1994, The Journal of thoracic and cardiovascular surgery.