Measuring quality of care with routine data: avoiding confusion between performance indicators and health outcomes

Abstract Objective: To investigate the impact of factors outside the control of primary care on performance indicators proposed as measures of the quality of primary care. Design: Multiple regression analysis relating admission rates standardised for age and sex for asthma, diabetes, and epilepsy to socioeconomic population characteristics and to the supply of secondary care resources. Setting: 90 family health services authorities in England, 1989-90 to 1994-5. Results: At health authority level socioeconomic characteristics, health status, and secondary care supply factors explained 45% of the variation in admission rates for asthma, 33% for diabetes, and 55% for epilepsy. When health authorities were ranked, only four of the 10 with the highest age-sex standardised admission rates for asthma in 1994-5 remained in the top 10 when allowance was made for socioeconomic characteristics, health status, and secondary care supply factors. There was also substantial year to year variation in the rates. Conclusion: Health outcomes should relate to crude rates of adverse events in the population. These give the best indication of the size of a health problem. Performance indicators, however, should relate to those aspects of care which can be altered by the staff whose performance is being measured. Key messages The NHS executive has proposed that admission rates for asthma, diabetes, and epilepsy could be used at health authority level as indicators of the quality of primary care There is considerable year to year variation in the ranking of health authorities by admission rates for these conditions, even when rates are aggregated. This makes it hard to interpret a single year's data: a 3 year average would be more reliable Morbidity, socioeconomic characteristics, and secondary care supply are important confounding factors that explain between a third and a half of the variation in admission rates across health authority areas Performance indicators should relate to aspects of care that can be controlled by decision makers. Confounding factors have a clear impact on admission rates and must therefore be taken into account if such rates are to be used as performance indicators

[1]  Peter C. Fishburn,et al.  Ordinal Preferences and Uncertain Lifetimes , 1978 .

[2]  D. Spiegelhalter,et al.  Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. , 1998, BMJ.

[3]  C. Griffiths,et al.  Hospital admissions for asthma in east london: associations with characteristics of local general practices, prescribing, and population , 1997, BMJ.

[4]  D. Plamping,et al.  The new NHS. , 1991, BMJ.

[5]  R. Klein,et al.  Explaining outputs of primary health care: population and practice factors. , 1991, BMJ.

[6]  G. DeFriese,et al.  The effects of having a regular doctor on access to primary care. , 1996, Medical care.

[7]  Stuart Peacock,et al.  A formula for distributing NHS revenues based on small area use of hospital beds , 1994 .

[8]  A. Farmer,et al.  Organization of care for diabetic patients in general practice: influence on hospital admissions. , 1990, The British journal of general practice : the journal of the Royal College of General Practitioners.

[9]  James G. MacKinnon,et al.  TESTS FOR MODEL SPECIFICATION IN THE PRESENCE OF ALTERNATIVE HYPOTHESES Some Further Results , 1983 .

[10]  H. White.  A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[11]  C Gatsonis,et al.  Rates of avoidable hospitalization by insurance status in Massachusetts and Maryland. , 1992, JAMA.

[12]  J. Billings,et al.  Recent findings on preventable hospitalizations. , 1996, Health affairs.

[13]  V. Hasselblad,et al.  A preliminary study: Use of small area analysis to assess the performance of the outpatient delivery system in New York City , 1989 .

[14]  R. Lewis,et al.  The relationship between asthma admission rates, routes of admission, and socioeconomic deprivation. , 1996, The European respiratory journal.

[15]  R. Madeley,et al.  Improved primary care does not prevent the admission of children to hospital. , 1989, Public health.

[16]  M. Roland,et al.  How much variation in referral rates among general practitioners is due to chance? , 1989, BMJ.

[17]  P Aveyard,et al.  Monitoring the performance of general practices. , 1997, Journal of evaluation in clinical practice.

[18]  A. Stewart,et al.  Preventable hospitalizations and access to health care. , 1995, JAMA.

[19]  J Gabbay,et al.  Performance indicators for primary care groups: an evidence based approach , 1998, BMJ.

[20]  B. Starfield,et al.  Hospitalizations of Children and Access to Primary Care: A Cross-National Comparison , 1995, International journal of health services : planning, administration, evaluation.