Control chart methods for monitoring surgical performance: a case study from gastro-oesophageal surgery.

Graphical methods are becoming increasingly used to monitor adverse outcomes from surgical interventions. However, uptake of such methods has largely been in the area of cardiothoracic surgery or in transplants with relatively little impact made in surgical oncology. A number of the more commonly used graphical methods including the Cumulative Mortality plot, Variable Life-Adjusted Display, Cumulative Sum (CUSUM) and funnel plots will be described. Accounting for heterogeneity in case-mix will be discussed and how ignoring case-mix can have considerable consequences. All methods will be illustrated using data from the Scottish Audit of Gastro-Oesophageal Cancer services (SAGOCS) data set.

[1]  M. Caputo,et al.  Control chart methods for monitoring cardiac surgical performance and their interpretation. , 2004, The Journal of thoracic and cardiovascular surgery.

[2]  J. Steurer,et al.  Multidimensional Analysis of Learning Curves in Laparoscopic Sigmoid Resection , 2003, Diseases of the colon and rectum.

[3]  Ewout W Steyerberg,et al.  Long-term survival after non-small cell lung cancer surgery: development and validation of a prognostic model with a preoperative and postoperative mode. , 2006, The Journal of thoracic and cardiovascular surgery.

[4]  S. Nashef,et al.  Perception and Reporting of Cardiac Surgical Performance , 2008, Seminars in cardiothoracic and vascular anesthesia.

[5]  D. Watters,et al.  TIME TO CUSUM: SIMPLIFIED REPORTING OF OUTCOMES IN COLORECTAL SURGERY , 2007, ANZ journal of surgery.

[6]  William H. Woodall,et al.  The Use of Control Charts in Health-Care and Public-Health Surveillance , 2006 .

[7]  R J Cook,et al.  Risk-Adjusted Monitoring of Binary Surgical Outcomes , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: validating a prognostic model , 2009, BMJ : British Medical Journal.

[9]  Eugene H Blackstone,et al.  Monitoring surgical performance. , 2004, The Journal of thoracic and cardiovascular surgery.

[10]  C. Delaney,et al.  Evaluation of the Learning Curve in Laparoscopic Colorectal Surgery: Comparison of Right-Sided and Left-Sided Resections , 2005, Annals of surgery.

[11]  B Arnrich,et al.  On-line variable live-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery. , 2004, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[12]  Measuring surgeon performance of sentinel lymph node biopsy in breast cancer treatment by cumulative sum analysis. , 2007, American journal of surgery.

[13]  Luc Noyez,et al.  Control charts, Cusum techniques and funnel plots. A review of methods for monitoring performance in healthcare. , 2009, Interactive cardiovascular and thoracic surgery.

[14]  E. S. Page.  CONTINUOUS INSPECTION SCHEMES , 1954 .

[15]  G. Grunkemeier,et al.  Cumulative sum techniques for assessing surgical results. , 2003, The Annals of thoracic surgery.

[16]  Ara W. Darzi,et al.  Funnel Plots and Their Emerging Application in Surgery , 2009, Annals of surgery.

[17]  A. Bernstein,et al.  A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. , 1989, Circulation.

[18]  J. M. East,et al.  Sentinel lymph node biopsy for breast cancer using methylene blue dye manifests a short learning curve among experienced surgeons: a prospective tabular cumulative sum (CUSUM) analysis , 2009, BMC surgery.

[19]  P. Mcculloch,et al.  ASCOT: a comprehensive clinical database for gastro-oesophageal cancer surgery. , 2001, European journal of surgical oncology : the journal of the European Society of Surgical Oncology and the British Association of Surgical Oncology.

[20]  Dong Hyun Choi,et al.  Learning curves for laparoscopic sigmoidectomy used to manage curable sigmoid colon cancer: single-institute, three-surgeon experience , 2009, Surgical Endoscopy.

[21]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: Developing a prognostic model , 2009, BMJ : British Medical Journal.

[22]  Dave Collett,et al.  The UK Scheme for Mandatory Continuous Monitoring of Early Transplant Outcome in all Kidney Transplant Centers , 2009, Transplantation.

[23]  Stefan Steiner,et al.  Direct comparison of risk-adjusted and non-risk-adjusted CUSUM analyses of coronary artery bypass surgery outcomes. , 2006, The Journal of thoracic and cardiovascular surgery.

[24]  Jeremy L. Ward,et al.  Mortality and morbidity in gastro-oesophageal cancer surgery: initial results of ASCOT multicentre prospective cohort study , 2003, BMJ : British Medical Journal.

[25]  Tom Treasure,et al.  Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[26]  Chris Sherlaw-Johnson,et al.  Monitoring the results of cardiac surgery by variable life-adjusted display , 1997, The Lancet.

[27]  David J Spiegelhalter,et al.  Funnel plots for comparing institutional performance , 2005, Statistics in medicine.

[28]  Y. Vergouwe,et al.  Validation, updating and impact of clinical prediction rules: a review. , 2008, Journal of clinical epidemiology.

[29]  T. Rapson,et al.  Hospital volume does not influence long‐term survival of patients undergoing surgery for oesophageal or gastric cancer , 2007, The British journal of surgery.

[30]  Nokuthaba Sibanda,et al.  The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data , 2007, BMC medical research methodology.

[31]  G. P. Copeland,et al.  POSSUM: A scoring system for surgical audit , 1991, The British journal of surgery.

[32]  A. Morton,et al.  Cumulative sum control charts for assessing performance in arterial surgery , 2004, ANZ journal of surgery.

[33]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[34]  Mats Brommels,et al.  Application of statistical process control in healthcare improvement: systematic review , 2007, Quality and Safety in Health Care.

[35]  C. Earle,et al.  Surgical mortality in patients with esophageal cancer: development and validation of a simple risk score. , 2006, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[36]  Steve Gallivan,et al.  Surgical Performance Measurement , 2002, Health care management science.

[37]  Martin Bland,et al.  Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups , 2004, BMJ : British Medical Journal.

[38]  R M Merion,et al.  Transplant Center Quality Assessment Using a Continuously Updatable, Risk‐Adjusted Technique (CUSUM) , 2006, American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons.

[39]  Joel Dunning,et al.  Funnel plots for comparing performance of PCI performing hospitals and cardiologists: Demonstration of utility using the New York hospital mortality data , 2009, Catheterization and cardiovascular interventions : official journal of the Society for Cardiac Angiography & Interventions.

[40]  Martin Utley,et al.  Monitoring risk-adjusted outcomes in congenital heart surgery: does the appropriateness of a risk model change with time? , 2009, The Annals of thoracic surgery.

[41]  Z Alfirevic,et al.  Monitoring obstetricians’ performance with statistical process control charts , 2007, BJOG : an international journal of obstetrics and gynaecology.

[42]  J. Dunning,et al.  Cumulative funnel plots for the early detection of interoperator variation: retrospective database analysis of observed versus predicted results of percutaneous coronary intervention , 2008, BMJ : British Medical Journal.

[43]  V T Farewell,et al.  Use of risk-adjusted CUSUM and RSPRTcharts for monitoring in medical contexts , 2003, Statistical methods in medical research.

[44]  T. Treasure,et al.  The influence of the surgeon on outcome , 1999 .

[45]  D. Siassakos,et al.  Prospective evaluation of a continuous monitoring and quality-improvement system for reducing adverse neonatal outcomes. , 2009, American journal of obstetrics and gynecology.

[46]  Ronald B. Crosier,et al.  Fast Initial Response for CUSUM Quality-Control Schemes: Give Your CUSUM A Head Start.: Give Your CUSUM A Head Start. , 2000 .

[47]  Tom Marshall,et al.  Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons , 2001, The Lancet.

[48]  Graeme K Hart,et al.  Review of the application of risk-adjusted charts to analyse mortality outcomes in critical care. , 2008, Critical care and resuscitation : journal of the Australasian Academy of Critical Care Medicine.

[49]  Vern T. Farewell,et al.  An overview of risk‐adjusted charts , 2004 .