Monitoring surgical and medical outcomes: the Bernoulli cumulative SUM chart. A novel application to assess clinical interventions

Background: Monitoring clinical interventions is an increasing requirement in current clinical practice. The standard CUSUM (cumulative sum) charts are used for this purpose. However, they are difficult to use in terms of identifying the point at which outcomes begin to be outside recommended limits. Objective: To assess the Bernoulli CUSUM chart that permits not only a 100% inspection rate, but also the setting of average expected outcomes, maximum deviations from these, and false positive rates for the alarm signal to trigger. Methods: As a working example this study used 674 consecutive first liver transplant recipients. The expected one year mortality set at 24% from the European Liver Transplant Registry average. A standard CUSUM was compared with Bernoulli CUSUM: the control value mortality was therefore 24%, maximum accepted mortality 30%, and average number of observations to signal was 500—that is, likelihood of false positive alarm was 1:500. Results: The standard CUSUM showed an initial descending curve (nadir at patient 215) then progressively ascended indicating better performance. The Bernoulli CUSUM gave three alarm signals initially, with easily recognised breaks in the curve. There were no alarms signals after patient 143 indicating satisfactory performance within the criteria set. Conclusions: The Bernoulli CUSUM is more easily interpretable graphically and is more suitable for monitoring outcomes than the standard CUSUM chart. It only requires three parameters to be set to monitor any clinical intervention: the average expected outcome, the maximum deviation from this, and the rate of false positive alarm triggers.

[1]  Martin Bland,et al.  Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups , 2004, BMJ : British Medical Journal.

[2]  M. Mckee,et al.  Hospital mortality league tables , 2003, BMJ : British Medical Journal.

[3]  S M Williams,et al.  Quality control: an application of the cusum. , 1992, BMJ.

[4]  Zachary G. Stoumbos,et al.  A CUSUM Chart for Monitoring a Proportion When Inspecting Continuously , 1999 .

[5]  M. Dean Hospital mortality league tables. , 1994, Lancet.

[6]  G Gallus,et al.  On surveillance methods for congenital malformations. , 1986, Statistics in medicine.

[7]  T. Treasure,et al.  Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective study , 2003, BMJ : British Medical Journal.

[8]  V T Farewell,et al.  Use of risk-adjusted CUSUM and RSPRTcharts for monitoring in medical contexts , 2003, Statistical methods in medical research.

[9]  G. Teasdale Learning from Bristol: report of the public inquiry into children's heart surgery at Bristol Royal Infirmary 1984-1995 , 2002, British journal of neurosurgery.

[10]  Shan Cretin,et al.  How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience , 2003, BMJ : British Medical Journal.

[11]  S. Nashef,et al.  The logistic EuroSCORE , 2003 .

[12]  Jones Mark,et al.  Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data , 2003, BMJ : British Medical Journal.

[13]  M. Kolczak,et al.  Monitoring mortality events associated with individual physicians and practices , 2003, The Lancet.

[14]  N. Best,et al.  Following Shipman: a pilot system for monitoring mortality rates in primary care , 2003, The Lancet.

[15]  P. Neuhaus,et al.  Normalised intrinsic mortality risk in liver transplantation: European Liver Transplant Registry study , 2000, The Lancet.

[16]  Neil C. Schwertman,et al.  OPTIMAL LIMITS FOR ATTRIBUTES CONTROL CHARTS , 1997 .

[17]  William H. Woodall,et al.  Control Charts Based on Attribute Data: Bibliography and Review , 1997 .

[18]  Tom Treasure,et al.  Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[19]  E. Blackstone,et al.  Can the outcome of coronary bypass grafting be predicted reliably? , 1997, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[20]  S. Lemeshow,et al.  European system for cardiac operative risk evaluation (EuroSCORE). , 1999, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[21]  P. Sergeant,et al.  La critique est aisee mais I'art est difficile , 1997, The Lancet.

[22]  Marion R. Reynolds,et al.  Control charts applying a general sequential test at each sampling point , 1996 .

[23]  Peter McCulloch,et al.  Jan using hospital mortality data performance of surgical units : validation study Mortality control charts for comparing , 2003 .

[24]  Chris Sherlaw-Johnson,et al.  Monitoring the results of cardiac surgery by variable life-adjusted display , 1997, The Lancet.

[25]  O. Dyer Heart surgeons are to be rated according to bypass surgery success , 2003, BMJ : British Medical Journal.

[26]  L. Grande,et al.  Quality control of the European Liver Transplant Registry: results of audit visits to the contributing centers , 2003, Transplantation.

[27]  D G Altman,et al.  The hidden effect of time. , 1988, Statistics in medicine.

[28]  S. Steiner,et al.  Monitoring surgical performance using risk-adjusted cumulative sum charts. , 2000, Biostatistics.

[29]  K. Detre,et al.  The relationship between outcome of liver transplantation and experience in new centers. , 1995, Liver transplantation and surgery : official publication of the American Association for the Study of Liver Diseases and the International Liver Transplantation Society.

[30]  B. Bridgewater,et al.  Limitations of the Parsonnet score for measuring risk stratified mortality in the north west of England , 2000, Heart.