Calculations of Financial Incentives for Providers in a Pay-for-Performance Program: Manual Review Versus Data From Structured Fields in Electronic Health Records

Background:Hospital report cards and financial incentives linked to performance require clinical data that are reliable, appropriate, timely, and cost-effective to process. Pay-for-performance plans are transitioning to automated electronic health record (EHR) data as an efficient method to generate data needed for these programs. Objective:To determine how well data from automated processing of structured fields in the electronic health record (AP-EHR) reflect data from manual chart review and the impact of these data on performance rewards. Research Design:Cross-sectional analysis of performance measures used in a cluster randomized trial assessing the impact of financial incentives on guideline-recommended care for hypertension. Subjects:A total of 2840 patients with hypertension assigned to participating physicians at 12 Veterans Affairs hospital-based outpatient clinics. Fifty-two physicians and 33 primary care personnel received incentive payments. Measures:Overall, positive and negative agreement indices and Cohen’s kappa were calculated for assessments of guideline-recommended antihypertensive medication use, blood pressure (BP) control, and appropriate response to uncontrolled BP. Pearson’s correlation coefficient was used to assess how similar participants’ calculated earnings were between the data sources. Results:By manual chart review data, 72.3% of patients were considered to have received guideline-recommended antihypertensive medications compared with 65.0% by AP-EHR review (&kgr;=0.51). Manual review indicated 69.5% of patients had controlled BP compared with 66.8% by AP-EHR review (&kgr;=0.87). Compared with 52.2% of patients per the manual review, 39.8% received an appropriate response by AP-EHR review (&kgr;=0.28). Participants’ incentive payments calculated using the 2 methods were highly correlated (r≥0.98). Using the AP-EHR data to calculate earnings, participants’ payment changes ranged from a decrease of $91.00 (−30.3%) to an increase of $18.20 (+7.4%) for medication use (interquartile range, −14.4% to 0%) and a decrease of $100.10 (−31.4%) to an increase of $36.40 (+15.4%) for BP control or appropriate response to uncontrolled BP (interquartile range, −11.9% to −6.1%). Conclusions:Pay-for-performance plans that use only EHR data should carefully consider the measures and the structure of the EHR before data collection and financial incentive disbursement. For this study, we feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared with manual review is acceptable given the time and resources required to abstract data from medical records.

[1]  Julia Adler-Milstein,et al.  More than half of US hospitals have at least a basic EHR, but stage 2 criteria remain challenging for most. , 2014, Health affairs.

[2]  William Gardner,et al.  Using computer-extracted data from electronic health records to measure the quality of adolescent well-care. , 2014, Health services research.

[3]  J. Rumsfeld,et al.  Insights from advanced analytics at the Veterans Health Administration. , 2014, Health affairs.

[4]  Terhilda Garrido,et al.  e-Measures: insight into the challenges and opportunities of automating publicly reported quality measures , 2013, J. Am. Medical Informatics Assoc..

[5]  Hhs Centers for Medicare Medicaid Services,et al.  Medicare and Medicaid programs: hospital outpatient prospective payment and ambulatory surgical center payment systems and quality reporting programs; Hospital Value-Based Purchasing Program; organ procurement organizations; quality improvement organizations; Electronic Health Records (EHR) Incentiv , 2013, Federal register.

[6]  T. Urech,et al.  Effects of individual physician-level and practice-level financial incentives on hypertension care: a randomized trial. , 2013, JAMA.

[7]  R. Kaushal,et al.  Accuracy of Electronically Reported “Meaningful Use” Clinical Quality Measures , 2013, Annals of Internal Medicine.

[8]  Sameer Malhotra,et al.  Accuracy of Electronically Reported “Meaningful Use” Clinical Quality Measures , 2013, Annals of Internal Medicine.

[9]  K. Pietz,et al.  Financial incentives and physician commitment to guideline-recommended hypertension management. , 2012, American Journal of Managed Care.

[10]  Jason Wang,et al.  Validity of electronic health record-derived quality measurement for performance monitoring , 2012, J. Am. Medical Informatics Assoc..

[11]  T. Urech,et al.  Design, rationale, and baseline characteristics of a cluster randomized controlled trial of pay for performance for hypertension treatment: study protocol , 2011, Implementation science : IS.

[12]  Patient self-report and medical records: measuring agreement for binary data. , 2011, Canadian family physician Medecin de famille canadien.

[13]  Hude Quan,et al.  Measuring agreement of administrative data with chart data using prevalence unadjusted and adjusted kappa , 2009, BMC medical research methodology.

[14]  Stephen D. Persell,et al.  Automated Review of Electronic Health Records to Assess Quality of Care for Outpatients with Heart Failure , 2007, Annals of Internal Medicine.

[15]  Stephen D. Persell,et al.  Assessing the validity of national quality measures for coronary artery disease using an electronic health record. , 2006, Archives of internal medicine.

[16]  D. Berlowitz,et al.  Can we use automated data to assess quality of hypertension care? , 2004, The American journal of managed care.

[17]  G. Joseph,et al.  Pharmacy Data in the VA Health Care System , 2003, Medical care research and review : MCRR.

[18]  Daniel W. Jones,et al.  The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: the JNC 7 report. , 2003, JAMA.

[19]  Steven H. Brown,et al.  VistA - U.S. Department of Veterans Affairs national-scale HIS , 2003, Int. J. Medical Informatics.

[20]  A. Feinstein,et al.  High agreement but low kappa: II. Resolving the paradoxes. , 1990, Journal of clinical epidemiology.

[21]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[22]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .