Unintended Consequences of Implementing a National Performance Measurement System into Local Practice

ABSTRACTBACKGROUNDAlthough benefits of performance measurement (PM) systems have been well documented, there is little research on negative unintended consequences of performance measurement systems in primary care. To optimize PM systems, a better understanding is needed of the types of negative unintended consequences that occur and of their causal antecedents.OBJECTIVES(1) Identify unintended negative consequences of PM systems for patients. (2) Develop a conceptual framework of hypothesized relationships between PM systems, facility-level variables (local implementation strategies, primary care staff attitudes and behaviors), and unintended negative effects on patients.DESIGN, PARTICIPANTS, APPROACHQualitative study design using dissimilar cases sampling. A series of 59 in-person individual semi-structured interviews at four Veterans Health Administration (VHA) facilities was conducted between February and July 2009. Participants included members of primary care staff and facility leaders. Sites were selected to assure variability in the number of veterans served and facility scores on national VHA performance measures. Interviews were recorded, transcribed and content coded to identify thematic categories and relationships.RESULTSParticipants noted both positive effects and negative unintended consequences of PM. We report three negative unintended consequences for patients. Performance measurement can (1) lead to inappropriate clinical care, (2) decrease provider focus on patient concerns and patient service, and (3) compromise patient education and autonomy. We also illustrate examples of negative consequences on primary care team dynamics. In many instances these problems originate from local implementation strategies developed in response to national PM definitions and policies.CONCLUSIONSFacility-level strategies undertaken to implement national PM systems may result in inappropriate clinical care, can distract providers from patient concerns, and may have a negative effect on patient education and autonomy. Further research is needed to ascertain how features of centralized PM systems influence whether measures are translated locally by facilities into more or less patient-centered policies and processes.

[1]  S. Mcphee,et al.  The Impact of Clinical Pharmacists' Consultations on Physicians' Geriatric Drug Prescribing: A Randomized Controlled Trial , 1992, Medical care.

[2]  K. Devers,et al.  Qualitative data analysis for health services research: developing taxonomy, themes, and theory. , 2007, Health services research.

[3]  D M Eddy,et al.  Performance measurement: problems and solutions. , 1998, Health affairs.

[4]  M. Rosenthal,et al.  Early experience with pay-for-performance: from concept to practice. , 2005, JAMA.

[5]  A. Bindman,et al.  Can physician profiles be trusted? , 1999, JAMA.

[6]  P. Shekelle,et al.  Comparison of Quality of Care for Patients in the Veterans Health Administration and Patients in a National Sample , 2004, Annals of Internal Medicine.

[7]  R. Hayward,et al.  Avoiding pitfalls in chronic disease quality measurement: a case for the next generation of technical quality measures. , 2001, The American journal of managed care.

[8]  A. Wu,et al.  Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. , 2005, JAMA.

[9]  T Sheldon,et al.  Promoting health care quality: what role performance indicators? , 1998, Quality in health care : QHC.

[10]  D. Asch,et al.  Racial Profiling: The Unintended Consequences of Coronary Artery Bypass Graft Report Cards , 2005, Circulation.

[11]  Sheila Roman,et al.  Public reporting and pay for performance in hospital quality improvement. , 2007, The New England journal of medicine.

[12]  A. Jha,et al.  Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. , 2003, The New England journal of medicine.

[13]  Dean F. Sittig,et al.  A survey of factors affecting clinician acceptance of clinical decision support , 2006, BMC Medical Informatics Decis. Mak..

[14]  P. Pronovost,et al.  Public Reporting of Antibiotic Timing in Patients with Pneumonia: Lessons from a Flawed Performance Measure , 2008, Annals of Internal Medicine.

[15]  R. Sulkava,et al.  Polypharmacy Status as an Indicator of Mortality in an Elderly Population , 2009, Drugs & aging.

[16]  Colin Sharp Qualitative Research and Evaluation Methods (3rd ed.) , 2003 .

[17]  John Gabbay,et al.  Reactions to the use of evidence-based performance indicators in primary care: a qualitative study , 2000, Quality in health care : QHC.

[18]  Yujing Shen,et al.  Selection incentives in a performance-based contracting system. , 2003, Health services research.

[19]  H. Perry,et al.  Polypharmacy and hospitalization among older home care patients. , 2000, The journals of gerontology. Series A, Biological sciences and medical sciences.

[20]  A. Epstein Paying for performance in the United States and abroad. , 2006, The New England journal of medicine.

[21]  M. Kuzuya,et al.  Underappreciated Predictors for Postdischarge Mortality in Acute Hospitalized Oldest-Old Patients , 2006, Gerontology.

[22]  Rodney A Hayward,et al.  Building a Better Quality Measure: Are Some Patients With ‘Poor Quality’ Actually Getting Good Care? , 2003, Medical care.

[23]  Joseph White,et al.  Paying for performance in primary medical care: learning about and learning from "success" and "failure" in England and California. , 2009, Journal of health politics, policy and law.

[24]  M. Patton Qualitative research and evaluation methods , 1980 .

[25]  Robert M Wachter,et al.  Accountability measures--using measurement to promote quality improvement. , 2010, The New England journal of medicine.

[26]  J. Pugh,et al.  What VA Providers Really Think About Clinical Practice Guidelines , 2004 .

[27]  A. K. Geddes,et al.  The Child in America: Behavior Problems and Programs. , 1929 .

[28]  L. Snyder,et al.  Pay-for-performance principles that promote patient-centered care: an ethics manifesto. , 2007, Annals of internal medicine.

[29]  R. Hayward Performance measurement in search of a path. , 2007, The New England journal of medicine.

[30]  D Blumenthal,et al.  The vital role of professionalism in health care reform. , 1994, Health affairs.

[31]  R. Bobrow The unintended consequences of measuring quality on the quality of medical care. , 2000, The New England journal of medicine.

[32]  E. Fisher Paying for performance--risks and recommendations. , 2006, The New England journal of medicine.

[33]  Julie M Hook,et al.  Incentive Implementation in Physician Practices: A Qualitative Study of Practice Executive Perspectives on Pay for Performance , 2006, Medical care research and review : MCRR.

[34]  R. Palmer,et al.  Suboptimal medication use and mortality in an older adult community-based cohort: results from the Hispanic EPESE Study. , 2006, The journals of gerontology. Series A, Biological sciences and medical sciences.

[35]  Evangelos Kontopantelis,et al.  Pay-for-performance programs in family practices in the United Kingdom. , 2006, The New England journal of medicine.

[36]  B. Sibbald,et al.  Effects of pay for performance on the quality of primary care in England. , 2009, The New England journal of medicine.