Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery.

OBJECTIVE To investigate the use of the risk-adjusted sequential probability ratio test in monitoring the cumulative occurrence of adverse clinical outcomes. DESIGN Retrospective analysis of three longitudinal datasets. SUBJECTS Patients aged 65 years and over under the care of Harold Shipman between 1979 and 1997, patients under 1 year of age undergoing paediatric heart surgery in Bristol Royal Infirmary between 1984 and 1995, adult patients receiving cardiac surgery from a team of cardiac surgeons in London,UK. MAIN OUTCOME MEASURE Annual and 30-day mortality rates. RESULTS Using reasonable boundaries, the procedure could have indicated an 'alarm' in Bristol after publication of the 1991 Cardiac Surgical Register, and in 1985 or 1997 for Harold Shipman depending on the data source and the comparator. The cardiac surgeons showed no significant deviation from expected performance. CONCLUSIONS The risk-adjusted sequential probability test is simple to implement, can be applied in a variety of contexts, and might have been useful to detect specific instances of past divergent performance. The use of this and related techniques deserves further attention in the context of prospectively monitoring adverse clinical outcomes.

[1]  A. Wald.  Sequential Tests of Statistical Hypotheses , 1945 .

[2]  A F Mackintosh,et al.  Use of cumulative mortality data in patients with acute myocardial infarction for early detection of variation in clinical practice: observational study , 2001, BMJ : British Medical Journal.

[3]  G. Teasdale.  Learning from Bristol: report of the public inquiry into children's heart surgery at Bristol Royal Infirmary 1984-1995 , 2002, British journal of neurosurgery.

[4]  K McPherson,et al.  Statistics: the problem of examining accumulating data more than once. , 1974, The New England journal of medicine.

[5]  J. Holden,et al.  The shipman inquiry. , 2004, The British journal of general practice : the journal of the Royal College of General Practitioners.

[6]  M R de Leval,et al.  Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switch operations. , 1994, The Journal of thoracic and cardiovascular surgery.

[7]  Tom Marshall,et al.  Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons , 2001, The Lancet.

[8]  Nicky Best,et al.  Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984–96: was Bristol an outlier? , 2001, The Lancet.

[9]  S. Steiner,et al.  Monitoring surgical performance using risk-adjusted cumulative sum charts. , 2000, Biostatistics.

[10]  P. Armitage,et al.  Sequential tests in prophylactic and therapeutic trials. , 1954, The Quarterly journal of medicine.

[11]  Harvey Goldstein,et al.  League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance , 1996 .

[12]  S Gallivan,et al.  Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice? , 2000, Heart.

[13]  Chris Sherlaw-Johnson,et al.  Monitoring the results of cardiac surgery by variable life-adjusted display , 1997, The Lancet.

[14]  P Littlejohns,et al.  Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery , 1998, BMJ.

[15]  A F BARTHOLOMAY.  The sequential probability ratio test applied to the design of clinical experiments. , 1957, The New England journal of medicine.

[16]  C L Christiansen,et al.  Improving the Statistical Approach to Health Care Provider Profiling , 1997, Annals of Internal Medicine.