Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity.

CONTEXT There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied. OBJECTIVE To evaluate the degree to which hospital outcome ratings and potential eligibility for financial incentives are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke. DESIGN, SETTING, AND PATIENTS Data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127,950 fee-for-service Medicare beneficiaries with ischemic stroke who had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared. MAIN OUTCOMES MEASURES Model discrimination, hospital 30-day mortality outcome rankings, and value-based purchasing financial incentive categories. RESULTS Across the study population, the mean (SD) NIHSS score was 8.23 (8.11) (median, 5; interquartile range, 2-12). There were 18,186 deaths (14.5%) within the first 30 days, including 7430 deaths (5.8%) during the index hospitalization. The hospital mortality model with NIHSS scores had significantly better discrimination than the model without (C statistic, 0.864; 95% CI, 0.861-0.867, vs 0.772; 95% CI, 0.769-0.776; P < .001). Among hospitals ranked in the top 20% or bottom 20% of performers by the claims model without NIHSS scores, 26.3% were ranked differently by the model with NIHSS scores. Of hospitals initially classified as having "worse than expected" mortality, 57.7% were reclassified to "as expected" by the model with NIHSS scores. The net reclassification improvement (93.1%; 95% CI, 91.6%-94.6%; P < .001) and integrated discrimination improvement (15.0%; 95% CI, 14.6%-15.3%; P < .001) indexes both demonstrated significant enhancement of model performance after the addition of NIHSS. Explained variance and model calibration was also improved with the addition of NIHSS scores. CONCLUSION Adding stroke severity as measured by the NIHSS to a hospital 30-day risk model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with considerably improved model discrimination and change in mortality performance rankings for a substantial portion of hospitals.

[1]  Gerhard Schroth,et al.  Predictors of early mortality after acute ischaemic stroke. , 2010, Swiss medical weekly.

[2]  C. Yancy,et al.  Incremental Value of Clinical Data Beyond Claims Data in Predicting 30-Day Outcomes After Heart Failure Hospitalization , 2011, Circulation. Cardiovascular quality and outcomes.

[3]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[4]  D. Mozaffarian,et al.  Heart disease and stroke statistics--2011 update: a report from the American Heart Association. , 2011, Circulation.

[5]  Vl Roger,et al.  American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart disease and stroke statistics-2011 update : a report from the American Heart Association , 2011 .

[6]  Hhs Centers for Medicare Medicare Services.  Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and FY 2012 rates; hospitals' FTE resident caps for graduate medical education payment. Final rules. , 2011, Federal register.

[7]  Eric E. Smith,et al.  Representativeness of the Get With The Guidelines–Stroke Registry: Comparison of Patient and Hospital Characteristics Among Medicare Beneficiaries Hospitalized With Ischemic Stroke , 2012, Stroke.

[8]  Harlan M Krumholz,et al.  An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With Heart Failure , 2006, Circulation.

[9]  Hhs Centers for Medicare Medicare Services.  Medicare program; hospital inpatient value-based purchasing program. Final rule. , 2011, Federal register.

[10]  S. Normand,et al.  Measuring performance for treating heart attacks and heart failure: the case for outcomes measurement. , 2007, Health affairs.

[11]  Eric E. Smith,et al.  Risk Score for In-Hospital Ischemic Stroke Mortality Derived and Validated Within the Get With The Guidelines–Stroke Program , 2010, Circulation.

[12]  Eric E. Smith,et al.  Age-Related Differences in Characteristics, Performance Measures, Treatment Trends, and Outcomes in Patients With Ischemic Stroke , 2010, Circulation.

[13]  A. Jha,et al.  Care in U.S. hospitals--the Hospital Quality Alliance program. , 2005, The New England journal of medicine.

[14]  A. Ziegler,et al.  Age and National Institutes of Health Stroke Scale Score Within 6 Hours After Onset Are Accurate Predictors of Outcome After Cerebral Ischemia: Development and External Validation of Prognostic Models , 2003, Stroke.

[15]  F. Dominici,et al.  Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[16]  Harlan M Krumholz,et al.  The performance of US hospitals as reflected in risk-standardized 30-day mortality and readmission rates for medicare beneficiaries with pneumonia. , 2010, Journal of hospital medicine.

[17]  D. Bravata,et al.  Metrics for Measuring Quality of Care in Comprehensive Stroke Centers: Detailed Follow-Up to Brain Attack Coalition Comprehensive Stroke Center Recommendations: A Statement for Healthcare Professionals From the American Heart Association/American Stroke Association , 2011, Stroke.

[18]  Adrian F Hernandez,et al.  Linking inpatient clinical registry data to Medicare claims data using indirect identifiers. , 2009, American heart journal.

[19]  M. Caldwell,et al.  American Heart Association: Council on Cardiovascular Nursing , 2004, The Journal of cardiovascular nursing.

[20]  S. Normand,et al.  Public reporting of 30-day mortality for patients hospitalized with acute myocardial infarction and heart failure. , 2008, Circulation.

[21]  L. Goldstein,et al.  Outcomes after ischemic stroke for hospitals with and without Joint Commission–certified primary stroke centers , 2011, Neurology.

[22]  J. Stoker,et al.  The Department of Health and Human Services. , 1999, Home healthcare nurse.

[23]  C. Benesch,et al.  Public Reporting of Quality Data for Stroke: Is It Measuring Quality? , 2008, Stroke.

[24]  Nancy R Cook,et al.  Advances in Measuring the Effect of Individual Predictors of Cardiovascular Risk: The Role of Reclassification Measures , 2009, Annals of Internal Medicine.

[25]  L. Goldstein,et al.  Predictors of Hospital Readmission After Stroke: A Systematic Review , 2010, Stroke.

[26]  Joseph P. Broderick,et al.  Relationship of National Institutes of Health Stroke Scale to 30-Day Mortality in Medicare Beneficiaries With Acute Ischemic Stroke , 2012, Journal of the American Heart Association.

[27]  W R Clarke,et al.  Baseline NIH Stroke Scale score strongly predicts outcome after stroke , 1999, Neurology.

[28]  Eric E. Smith,et al.  Hospital-Level Variation in Mortality and Rehospitalization for Medicare Beneficiaries With Acute Ischemic Stroke , 2011, Stroke.