A randomized controlled trial of league tables and control charts as aids to health service decision-making.

OBJECTIVES Health service managers and other decision-makers are required to act on the basis of data. Little attention has been paid to the effects of data presentation on the decisions taken. This study uses a randomized controlled trial design to investigate the effects of two forms of data presentation--league tables and control charts--on health service decision-makers. METHODS Directors of public health in 122 health authorities in the UK were mailed three case studies and a questionnaire. The case studies showed data on variations in mortality by health service provider. The questionnaire asked them to indicate whether they would take action as a result of the data and to identify the health service providers on whom they would take action. Participants were randomly allocated to receive the same data in the form of ranked histograms (league tables) or control charts. MAIN OUTCOME MEASURE The percentage of participants who would take action on health service providers. RESULTS Fifty-seven questionnaires were returned. For each case study, respondents receiving data as league tables stated they would take action on significantly more health service providers than those receiving data as control charts: for the first case study, the percentages were 3.3% versus 1.8% (P < 0.001) for league tables and control charts, respectively; for the second case study, 15.9% versus 6.7% (P = 0.029), respectively; and for the third, 5.9% versus 0.7% (P = 0.002), respectively. Respondents receiving data as league tables were significantly more likely to request further information on case mix. CONCLUSIONS Compared with league tables, health service decision-makers identify fewer outliers for further action when performance data are presented as control charts. They also reduce the tendency to request further information. Using control charts rather than league tables for the routine presentation of comparative data would reduce over-investigation of unusual performance.

[1]  S. Pathak.  The way I see it: Working in the Caribbean in the gap between basic and higher surgical training , 2002 .

[2]  T. Walley,et al.  Same information, different decisions: the influence of evidence on the management of hypertension in the elderly. , 1996, The British journal of general practice : the journal of the Royal College of General Practitioners.

[3]  J. G. Hollands,et al.  The visual communication of risk. , 1999, Journal of the National Cancer Institute. Monographs.

[4]  T. Lim.  Statistical process control tools for monitoring clinical performance. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[5]  Tom Marshall,et al.  Performance league tables: the NHS deserves better , 2002, BMJ : British Medical Journal.

[6]  C. Palmer,et al.  Differences in mortality after fracture of hip: the East Anglian audit , 1995, BMJ.

[7]  G. Elwyn,et al.  Presenting risk information--a review of the effects of "framing" and other manipulations on patient outcomes. , 2001, Journal of health communication.

[8]  Peter McCulloch,et al.  Jan using hospital mortality data performance of surgical units : validation study Mortality control charts for comparing , 2003 .

[9]  W. Edwards Deming.  Sample design in business research , 1960 .

[10]  A. R. Crathorne,et al.  Economic Control of Quality of Manufactured Product. , 1933 .

[11]  A. Zauberman,et al.  The new economics , 1965 .

[12]  Tom Marshall,et al.  Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons , 2001, The Lancet.

[13]  T. Fahey,et al.  Evidence based purchasing: understanding results of clinical trials and systematic reviews , 1995, BMJ.

[14]  Benjamin M. Adams,et al.  Advanced Topics in Statistical Process Control : The Power of Shewhart's Charts , 1995 .

[15]  F. Mosteller,et al.  The Uses and Usefulness of Binomial Probability Paper , 1949 .

[16]  W. Edwards Deming,et al.  Out of the Crisis , 1982 .