Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry

The public inquiry into paediatric cardiac surgery at the Bristol Royal Infirmary commissioned the authors to design and conduct analyses of routine data sources to compare surgical outcomes between centres. Such analyses are necessarily complex in this context but were further hampered by the inherent inconsistencies and mediocre quality of the various sources of data. Three levels of analysis of increasing sophistication were carried out. The reasonable consistency of the results arising from different sources of data, together with a number of sensitivity analyses, led us to conclude that there had been excess mortality in Bristol in open heart operations on children under 1 year of age. We consider criticisms of our analysis and discuss the role of statisticians in this inquiry and their contribution to the final report of the inquiry. The potential statistical role in future programmes for monitoring clinical performance is highlighted.

[1]  G. Teasdale.  Learning from Bristol: report of the public inquiry into children's heart surgery at Bristol Royal Infirmary 1984-1995 , 2002, British journal of neurosurgery.

[2]  Emer O'hagan.  The European social model in economic peripheries , 2002 .

[3]  A. Dobson,et al.  The application of statistical process control charts to the detection and monitoring of hospital-acquired infections. , 2001, Journal of quality in clinical practice.

[4]  A. Hasan,et al.  Cardiopulmonary transplantation in children: a reason for optimism? , 2001, Heart.

[5]  H. Goldstein,et al.  Predicting the Future : the role of past performance in determining trends in institutional effectiveness at A level , 2001 .

[6]  Nicky Best,et al.  Comparison of UK paediatric cardiac surgical performance by analysis of routinely collected data 1984–96: was Bristol an outlier? , 2001, The Lancet.

[7]  Tom Marshall,et al.  Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons , 2001, The Lancet.

[8]  M. J. Bayarri,et al.  P Values for Composite Null Models , 2000 .

[9]  R. Gibberd,et al.  Using clinical indicators to identify areas for quality improvement. , 2000, Journal of quality in clinical practice.

[10]  S. Steiner,et al.  Monitoring surgical performance using risk-adjusted cumulative sum charts. , 2000, Biostatistics.

[11]  C L Christiansen,et al.  Medical profiling: improving standards and risk adjustments using hierarchical models. , 2000, Journal of health economics.

[12]  S. Gallivan.  Overview of the statistical evidence presented to the Bristol Royal Infirmary Inquiry concerning the nature and outcomes of paediatric cardiac surgical services at Bristol relative to other specialist centres from 1984 to 1995 (Spiegelhalter, Evans, Aylin and Murray) , 2000 .

[13]  L I Iezzoni,et al.  Explaining differences in English hospital death rates using routinely collected data , 1999, BMJ.

[14]  Bob Gann,et al.  Information for Health , 1999, Health expectations : an international journal of public participation in health care and health policy.

[15]  David J. Spiegelhalter,et al.  Surgical audit: statistical lessons from Nightingale and Codman , 1999 .

[16]  J W Thomas,et al.  Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care. , 1999, Medical care.

[17]  I. Deary.  Differences in mental abilities , 1998, BMJ.

[18]  E. Rich,et al.  Atrial Flutter and Fibrillation: From Basic to Clinical Applications , 1998, Annals of Internal Medicine.

[19]  D. Spiegelhalter,et al.  Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. , 1998, BMJ.

[20]  P Littlejohns,et al.  Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery , 1998, BMJ.

[21]  E. Hannan,et al.  Pediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality. , 1998, Pediatrics.

[22]  Chris Sherlaw-Johnson,et al.  Monitoring the results of cardiac surgery by variable life-adjusted display , 1997, The Lancet.

[23]  C L Christiansen,et al.  Improving the Statistical Approach to Health Care Provider Profiling , 1997, Annals of Internal Medicine.

[24]  M. Glickman,et al.  Statistical Methods for Profiling Providers of Medical Care: Issues and Applications , 1997 .

[25]  Michael L. Millenson.  Demanding medical excellence , 1997 .

[26]  A M Epstein,et al.  Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. , 1996, The New England journal of medicine.

[27]  Harvey Goldstein,et al.  League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance , 1996 .

[28]  Thomas A. Louis,et al.  League tables and their limitations: Statistical issues in comparisons of institutional performance - Discussion , 1996 .

[29]  Xiao-Li Meng,et al.  POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES , 1996 .

[30]  M R de Leval,et al.  Analysis of a cluster of surgical failures. Application to a series of neonatal arterial switch operations. , 1994, The Journal of thoracic and cardiovascular surgery.

[31]  R Peto,et al.  Effects of intravenous magnesium in suspected acute myocardial infarction: overview of randomised trials. , 1991, BMJ.

[32]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[33]  D M Berwick,et al.  Continuous improvement as an ideal in health care. , 1989, The New England journal of medicine.

[34]  R. West,et al.  An example of medical audit under political pressure. , 1983, International Journal of Epidemiology.

[35]  W. Edwards Deming,et al.  Out of the Crisis , 1982 .

[36]  R. F.,et al.  Statistical Method from the Viewpoint of Quality Control , 1940, Nature.

[37]  F. Nightingale,et al.  Notes on Hospitals , 1860, The British and foreign medico-chirurgical review.