Process monitoring in intensive care with the use of cumulative expected minus observed mortality and risk-adjusted p charts

IntroductionA health care system is a complex adaptive system. The effect of a single intervention, incorporated into a complex clinical environment, may be different from that expected. A national database such as the Intensive Care National Audit & Research Centre (ICNARC) Case Mix Programme in the UK represents a centralised monitoring, surveillance and reporting system for retrospective quality and comparative audit. This can be supplemented with real-time process monitoring at a local level for continuous process improvement, allowing early detection of the impact of both unplanned and deliberately imposed changes in the clinical environment.MethodsDemographic and UK Acute Physiology and Chronic Health Evaluation II (APACHE II) data were prospectively collected on all patients admitted to a UK regional hospital between 1 January 2003 and 30 June 2004 in accordance with the ICNARC Case Mix Programme. We present a cumulative expected minus observed (E-O) plot and the risk-adjusted p chart as methods of continuous process monitoring. We describe the construction and interpretation of these charts and show how they can be used to detect planned or unplanned organisational process changes affecting mortality outcomes.ResultsFive hundred and eighty-nine adult patients were included. The overall death rate was 0.78 of predicted. Calibration showed excess survival in ranges above 30% risk of death. The E-O plot confirmed a survival above that predicted. Small transient variations were seen in the slope that could represent random effects, or real but transient changes in the quality of care. The risk-adjusted p chart showed several observations below the 2 SD control limits of the expected mortality rate. These plots provide rapid analysis of risk-adjusted performance suitable for local application and interpretation. The E-O chart provided rapid easily visible feedback of changes in risk-adjusted mortality, while the risk-adjusted p chart allowed statistical evaluation.ConclusionLocal analysis of risk-adjusted mortality data with an E-O plot and a risk-adjusted p chart is feasible and allows the rapid detection of changes in risk-adjusted outcome of intensive care patients. This complements the centralised national database, which is more archival and comparative in nature.

[1]  D. Cook,et al.  Prediction of risk of death using 30-day outcome: a practical end point for quality auditing in intensive care. , 2004, Chest.

[2]  S Gallivan,et al.  Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice? , 2000, Heart.

[3]  T. Osler,et al.  Effect of mortality rate on the performance of the Acute Physiology and Chronic Health Evaluation II: A simulation study , 2000, Critical care medicine.

[4]  A F Mackintosh,et al.  Use of cumulative mortality data in patients with acute myocardial infarction for early detection of variation in clinical practice: observational study , 2001, BMJ : British Medical Journal.

[5]  L. Iezzoni.  Risk Adjustment for Measuring Healthcare Outcomes , 1994 .

[6]  Tom Treasure,et al.  Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[7]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[8]  Stephen R. Marsland,et al.  Clinical Quality Needs Complex Adaptive Systems and Machine Learning , 2004, MedInfo.

[9]  S. Steiner,et al.  Monitoring surgical performance using risk-adjusted cumulative sum charts. , 2000, Biostatistics.

[10]  R J Cook,et al.  Risk-Adjusted Monitoring of Binary Surgical Outcomes , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[11]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[12]  Richard J Cook,et al.  Monitoring the evolutionary process of quality: risk-adjusted charting to track outcomes in intensive care. , 2003, Critical care medicine.

[13]  Kathy Rowan,et al.  Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: the Intensive Care National Audit & Research Centre Case Mix Programme Database , 2004, Critical care.

[14]  W H Chamberlin,et al.  Monitoring intensive care unit performance using statistical quality control charts , 1993, International journal of clinical monitoring and computing.

[15]  M. Caputo,et al.  Monitoring the performance of residents during training in off-pump coronary surgery. , 2004, The Journal of thoracic and cardiovascular surgery.

[16]  S Lemeshow,et al.  Factors affecting the performance of the models in the Mortality Probability Model II system and strategies of customization: a simulation study. , 1996, Critical care medicine.

[17]  Chris Sherlaw-Johnson,et al.  Real-time monitoring of coronary care mortality: a comparison and combination of two monitoring tools. , 2005, International journal of cardiology.

[18]  J H Kerr,et al.  Intensive Care Society's APACHE II study in Britain and Ireland--I: Variations in case mix of adult admissions to general intensive care units and impact on outcome. , 1993, BMJ.

[19]  J H Kerr,et al.  Intensive Care Society's Acute Physiology and Chronic Health Evaluation (APACHE II) study in Britain and Ireland: A prospective, multicenter, cohort study comparing two methods for predicting outcome for adult intensive care patients , 1994, Critical care medicine.

[20]  Martin Bland,et al.  Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups , 2004, BMJ : British Medical Journal.

[21]  Rachel Elliott,et al.  Case study eight: outcome comparisions of intensive care units in Great Britain and Ireland using the APACHE II method. , 1996 .

[22]  J. D. Young,et al.  Development and testing of a hierarchical method to code the reason for admission to intensive care units: the ICNARC Coding Method , 2001 .

[23]  S Lemeshow,et al.  Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. , 1996, Critical care medicine.

[24]  K. Rowan,et al.  Variation in intensive care unit outcomes: a search for the evidence on organizational factors , 2001, Current opinion in critical care.

[25]  Vern T. Farewell,et al.  An overview of risk‐adjusted charts , 2004 .

[26]  N. Banner,et al.  Cumulative risk adjusted monitoring of 30-day mortality after cardiothoracic transplantation: UK experience. , 2005, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[27]  T. Osler,et al.  Effect of varying the case mix on the standardized mortality ratio and W statistic: A simulation study. , 2000, Chest.

[28]  B. Taylor,et al.  The impact of low‐risk intensive care unit admissions on mortality probabilities by SAPS II, APACHE II and APACHE III , 2002, Anaesthesia.

[29]  J H Kerr,et al.  Intensive Care Society's APACHE II study in Britain and Ireland--II: Outcome comparisons of intensive care units after adjustment for case mix by the American APACHE II method. , 1993, BMJ.

[30]  E. Draper,et al.  APACHE II: A severity of disease classification system , 1985, Critical care medicine.

[31]  Chris Sherlaw-Johnson,et al.  Monitoring the results of cardiac surgery by variable life-adjusted display , 1997, The Lancet.

[32]  S. Hansson.  Dimensions of Risk , 1989 .

[33]  P Littlejohns,et al.  Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery , 1998, BMJ.

[34]  W J Sibbald,et al.  Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. , 1999, Critical care medicine.

[35]  V T Farewell,et al.  A risk‐adjusted Sets method for monitoring adverse medical outcomes , 2004, Statistics in medicine.

[36]  R J Cook,et al.  Monitoring paired binary surgical outcomes using cumulative sum charts. , 1999, Statistics in medicine.