Assessing the Accuracy of Hospital Clinical Performance Measures

Objective. To control costs and improve quality, payers are designing new hospital reimbursement policies that link payment to quality. The authors determine the extent to which quality measures discriminate hospitals into tiers in 2 geographic areas. Data Sources. Administrative and medical record data for patients discharged with acute myocardial infarction (AMI) in 368 California and 81 Massachusetts hospitals. Methods. Through simulation, the minimum numbers of patients per hospital needed to identify highquality hospitals with sensitivity ranging from 75% to 95% under a variety of clinical scenarios are determined. Results. Massachusetts hospitals had twice the number of eligible patients per hospital than California hospitals. Regardless of state, few hospitals had sufficient sample size needed to achieve >85% sensitivity for high-variation quality measures. Clinical Implications. Reliability of quality-based reimbursement systems relies on the distribution of the hospital sample sizes within geographic areas and the size of practice differences. Selection of conformance thresholds and sensitivity levels depends on the user of the information. To assess the usefulness of performance measures to tier hospitals, information regarding between-hospital variation in quality for specific clinical measures needs to be collected and reported.

[1]  J. Wennberg,et al.  Geographic variation in the treatment of acute myocardial infarction: the Cooperative Cardiovascular Project. , 1999, JAMA.

[2]  W. Manning,et al.  The Unreliability of Individual Physician “Report Cards” for Assessing the Costs and Quality of Care of a Chronic Disease , 2000 .

[3]  L. Leape,et al.  Comparison of Use of Medications After Acute Myocardial Infarction in the Veterans Health Administration and Medicare , 2001, Circulation.

[4]  S L Normand,et al.  Cardiac surgery report cards: comprehensive review and statistical critique. , 2001, The Annals of thoracic surgery.

[5]  A. Milstein,et al.  Large employers' new strategies in health care. , 2002, The New England journal of medicine.

[6]  K. Zou,et al.  Sample size considerations in observational health care quality studies , 2002, Statistics in medicine.

[7]  Sharon-Lise T. Normand,et al.  Analytic Methods for Constructing Cross-Sectional Profiles of Health Care Providers , 2000, Health Services and Outcomes Research Methodology.

[8]  Francois S de Brantes,et al.  A middle ground on public accountability. , 2004, The New England journal of medicine.

[9]  J. Birkmeyer,et al.  Surgical mortality as an indicator of hospital quality: the problem with small sample size. , 2004, JAMA.

[10]  R. Steinbrook.  The cost of admission--tiered copayments for hospital use. , 2004, The New England journal of medicine.

[11]  M. A. Best.  Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[12]  A. Jha,et al.  Care in U.S. hospitals--the Hospital Quality Alliance program. , 2005, The New England journal of medicine.