Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries.

IMPORTANCE The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) provides feedback to hospitals on risk-adjusted outcomes. It is not known if participation in the program improves outcomes and reduces costs relative to nonparticipating hospitals. OBJECTIVE To evaluate the association of enrollment and participation in the ACS NSQIP with outcomes and Medicare payments compared with control hospitals that did not participate in the program. DESIGN, SETTING, AND PARTICIPANTS Quasi-experimental study using national Medicare data (2003-2012) for a total of 1,226,479 patients undergoing general and vascular surgery at 263 hospitals participating in ACS NSQIP and 526 nonparticipating hospitals. A difference-in-differences analytic approach was used to evaluate whether participation in ACS NSQIP was associated with improved outcomes and reduced Medicare payments compared with nonparticipating hospitals that were otherwise similar. Control hospitals were selected using propensity score matching (2 control hospitals for each ACS NSQIP hospital). MAIN OUTCOMES AND MEASURES Thirty-day mortality, serious complications (eg, pneumonia, myocardial infarction, or acute renal failure and a length of stay >75th percentile), reoperation, and readmission within 30 days. Hospital costs were assessed using price-standardized Medicare payments during hospitalization and 30 days after discharge. RESULTS After accounting for patient factors and preexisting time trends toward improved outcomes, there were no statistically significant improvements in outcomes at 1, 2, or 3 years after (vs before) enrollment in ACS NSQIP. For example, in analyses comparing outcomes at 3 years after (vs before) enrollment, there were no statistically significant differences in risk-adjusted 30-day mortality (4.3% after enrollment vs 4.5% before enrollment; relative risk [RR], 0.96 [95% CI, 0.89 to 1.03]), serious complications (11.1% after enrollment vs 11.0% before enrollment; RR, 0.96 [95% CI, 0.91 to 1.00]), reoperations (0.49% after enrollment vs 0.45% before enrollment; RR, 0.97 [95% CI, 0.77 to 1.16]), or readmissions (13.3% after enrollment vs 12.8% before enrollment; RR, 0.99 [95% CI, 0.96 to 1.03]). There were also no differences at 3 years after (vs before) enrollment in mean total Medicare payments ($40 [95% CI, -$268 to $348]), or payments for the index admission (-$11 [95% CI, -$278 to $257]), hospital readmission ($245 [95% CI, -$231 to $721]), or outliers (-$86 [95% CI, -$1666 to $1495]). CONCLUSIONS AND RELEVANCE With time, hospitals had progressively better surgical outcomes but enrollment in a national quality reporting program was not associated with the improved outcomes or lower Medicare payments among surgical patients. Feedback on outcomes alone may not be sufficient to improve surgical outcomes.

[1]  L I Iezzoni,et al.  Identifying Complications of Care Using Administrative Data , 1994, Medical care.

[2]  E. Hannan,et al.  Provider Profiling and Quality Improvement Efforts in Coronary Artery Bypass Graft Surgery: The Effect on Short-Term Mortality Among Medicare Beneficiaries , 2003, Medical care.

[3]  J. Hibbard,et al.  Hospital performance reports: impact on quality, market share, and reputation. , 2005, Health affairs.

[4]  Jeannie Glickson.  ACS NSQIP National conference: speakers promote professionalism, collaboration to achieve quality improvement. , 2013, Bulletin of the American College of Surgeons.

[5]  M. Rosenthal,et al.  Beyond pay for performance--emerging models of provider-payment reform. , 2008, The New England journal of medicine.

[6]  W. Henderson,et al.  The National Surgical Quality Improvement Program in Non-Veterans Administration Hospitals: Initial Demonstration of Feasibility , 2002, Annals of surgery.

[7]  E John Orav,et al.  Variation in surgical-readmission rates and quality of hospital care. , 2013, The New England journal of medicine.

[8]  L I Iezzoni,et al.  Use of administrative data to find substandard care: validation of the complications screening program. , 2000, Medical care.

[9]  C. Steiner,et al.  Comorbidity measures for use with administrative data. , 1998, Medical care.

[10]  Jeffrey M. Wooldridge,et al.  Introductory Econometrics: A Modern Approach , 1999 .

[11]  J. Birkmeyer,et al.  Trends in hospital volume and operative mortality for high-risk surgery. , 2011, The New England journal of medicine.

[12]  Florence E. Turrentine,et al.  Use of national surgical quality improvement program data as a catalyst for quality improvement. , 2007, Journal of the American College of Surgeons.

[13]  C. Ko,et al.  Quality improvement in surgery: the American College of Surgeons National Surgical Quality Improvement Program approach. , 2010, Advances in surgery.

[14]  S. Ellner.  Hospital puts ACS NSQIP to the test and improves patient safety. , 2011, Bulletin of the American College of Surgeons.

[15]  J. Zhang,et al.  What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. , 1998, JAMA.

[16]  L I Iezzoni,et al.  Using administrative data to screen hospitals for high complication rates. , 1994, Inquiry : a journal of medical care organization, provision and financing.

[17]  J. Birkmeyer,et al.  Medicare payments for common inpatient procedures: implications for episode-based payment bundling. , 2010, Health services research.

[18]  David C. Miller,et al.  Large variations in Medicare payments for surgery highlight savings potential from bundled payment programs. , 2011, Health affairs.

[19]  Justin B Dimick,et al.  Variation in hospital mortality associated with inpatient surgery. , 2009, The New England journal of medicine.

[20]  A. Ryan.  Effects of the Premier Hospital Quality Incentive Demonstration on Medicare patient mortality and cost. , 2009, Health services research.

[21]  Hude Quan,et al.  Comparison of the Elixhauser and Charlson/Deyo Methods of Comorbidity Measurement in Administrative Data , 2004, Medical care.

[22]  C. Ko,et al.  Does Surgical Quality Improve in the American College of Surgeons National Surgical Quality Improvement Program: An Evaluation of All Participating Hospitals , 2009, Annals of surgery.

[23]  L. Neumayer,et al.  Using the Veterans Administration National Surgical Quality Improvement Program to improve patient outcomes. , 2000, The Journal of surgical research.

[24]  P. Austin,et al.  Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies , 2010, Pharmaceutical statistics.

[25]  J. Skinner,et al.  Prices don't drive regional Medicare spending variations. , 2010, Health affairs.

[26]  E. Copeland.  Who Pays for Poor Surgical Quality? Building a Business Case for Quality Improvement , 2007 .

[27]  J. Hibbard,et al.  It Isn't Just about Choice: The Potential of a Public Performance Report to Affect the Public Image of Hospitals , 2005, Medical care research and review : MCRR.

[28]  Karl Hammermeister,et al.  Successful Implementation of the Department of Veterans Affairs’ National Surgical Quality Improvement Program in the Private Sector: The Patient Safety in Surgery Study , 2008, Annals of surgery.

[29]  Robert H Brook,et al.  A Comparison of Clinical Registry Versus Administrative Claims Data for Reporting of 30-Day Surgical Complications , 2012, Annals of surgery.

[30]  M. Rosenthal,et al.  Nonpayment for performance? Medicare's new reimbursement rule. , 2007, The New England journal of medicine.

[31]  Ellen Meara,et al.  Spending differences associated with the Medicare Physician Group Practice Demonstration. , 2012, JAMA.

[32]  A. Morris,et al.  Reoperation as a Quality Indicator in Colorectal Surgery: A Population-Based Analysis , 2007, Annals of surgery.

[33]  E. Duflo,et al.  How Much Should We Trust Differences-in-Differences Estimates? , 2001 .

[34]  K Walshe,et al.  Effectiveness of quality improvement: learning from evaluations , 2002, BMJ Quality & Safety.

[35]  J. Birkmeyer,et al.  Prioritizing quality improvement in general surgery. , 2008, Journal of the American College of Surgeons.

[36]  Kwan Hur,et al.  Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. , 2002, Journal of the American College of Surgeons.

[37]  Kathryn E. McGoldrick,et al.  Surgeon Volume and Operative Mortality in the United States , 2004 .

[38]  Andrew M Ryan,et al.  Why We Should Not Be Indifferent to Specification Choices for Difference-in-Differences. , 2015, Health services research.

[39]  Stephen G. Donald,et al.  Inference with Difference-in-Differences and Other Panel Data , 2007, The Review of Economics and Statistics.

[40]  L I Iezzoni,et al.  Identification of in-hospital complications from claims data. Is it valid? , 2000, Medical care.

[41]  J. Birkmeyer,et al.  Prioritizing Quality Improvement in Vascular Surgery , 2010, Surgical Innovation.

[42]  E. Livingston.  Procedure incidence and in-hospital complication rates of bariatric surgery in the United States. , 2004, American journal of surgery.

[43]  J. Stockman.  Mortality Among Hospitalized Medicare Beneficiaries in the First 2 Years Following ACGME Resident Duty Hour Reform , 2009 .

[44]  Andrew M Ryan,et al.  Methods for evaluating changes in health care policy: the difference-in-differences approach. , 2014, JAMA.