Comparison of Clinical and Administrative Data Sources for Hospital Coronary Artery Bypass Graft Surgery Report Cards

Background— Regardless of statistical methodology, public performance report cards must use the highest-quality validated data, preferably from a prospectively maintained clinical database. Using logistic regression and hierarchical models, we compared hospital cardiac surgery profiling results based on clinical data with those derived from contemporaneous administrative data. Methods and Results— Fiscal year 2003 isolated coronary artery bypass grafting surgery results based on an audited and validated Massachusetts clinical registry were compared with those derived from a contemporaneous state administrative database, the latter using the inclusion/exclusion criteria and risk model of the Agency for Healthcare Research and Quality. There was a 27.4% disparity in isolated coronary artery bypass grafting surgery volume (4440 clinical, 5657 administrative), a 0.83% difference in observed in-hospital mortality (2.05% versus 2.88%), corresponding differences in risk-adjusted mortality calculated by various statistical methodologies, and 1 hospital classified as an outlier only with the administrative data-based approach. The discrepancies in volumes and risk-adjusted mortality were most notable for higher-volume programs that presumably perform a higher proportion of combined procedures that were misclassified as isolated coronary artery bypass grafting surgery in the administrative cohort. Subsequent analyses of a patient cohort common to both databases revealed the smoothing effect of hierarchical models, a 9% relative difference in mortality (2.21% versus 2.03%) resulting from nonstandardized mortality end points, and 1 hospital classified as an outlier using logistic regression but not using hierarchical regression. Conclusions— Cardiac surgery report cards using administrative data are problematic compared with those derived from audited and validated clinical data, primarily because of case misclassification and nonstandardized end points.

[1]  D. Carlisle,et al.  Administrative Versus Clinical Data for Coronary Artery Bypass Graft Surgery Report Cards: The View From California , 2006, Medical care.

[2]  S. Normand,et al.  An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With an Acute Myocardial Infarction , 2006, Circulation.

[3]  Harlan M Krumholz,et al.  An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With Heart Failure , 2006, Circulation.

[4]  T. Osler,et al.  Does date stamping ICD-9-CM codes increase the value of clinical information in administrative data? , 2006, Health services research.

[5]  Sharon-Lise T Normand,et al.  Massachusetts cardiac surgery report card: implications of statistical methodology. , 2005, The Annals of thoracic surgery.

[6]  S. Normand,et al.  Implementation of a cardiac surgery report card: lessons from the Massachusetts experience. , 2005, The Annals of thoracic surgery.

[7]  G. Meyer,et al.  Use of administrative data for clinical quality measurement. , 2005, The Journal of thoracic and cardiovascular surgery.

[8]  M. Mack,et al.  Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes? , 2005, The Journal of thoracic and cardiovascular surgery.

[9]  Nancy J. Petersen,et al.  Mortality After Cardiac Bypass Surgery: Prediction From Administrative Versus Clinical Data , 2005, Medical care.

[10]  Michael L. Johnson,et al.  Mortality After Noncardiac Surgery: Prediction From Administrative Versus Clinical Data , 2005, Medical care.

[11]  F. Grover,et al.  Cardiac surgery risk models: a position article. , 2004, The Annals of thoracic surgery.

[12]  D. Shahian.  Improving cardiac surgery quality--volume, outcome, process? , 2004, JAMA.

[13]  Asking too much of administrative data? , 2003, Journal of the American College of Surgeons.

[14]  J. Birkmeyer,et al.  Adjusting surgical mortality rates for patient comorbidities: more harm than good? , 2002, Surgery.

[15]  Patrick S Romano,et al.  Can Administrative Data Be Used to Compare Postoperative Complication Rates Across Hospitals? , 2002, Medical care.

[16]  Hude Quan,et al.  Validity of Information on Comorbidity Derived From ICD-9-CCM Administrative Data , 2002, Medical care.

[17]  H. Krumholz,et al.  Evaluation of a consumer-oriented internet health care report card: the risk of quality ratings based on mortality data. , 2002, JAMA.

[18]  Kwan Hur,et al.  Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. , 2002, Journal of the American College of Surgeons.

[19]  S L Normand,et al.  Cardiac surgery report cards: comprehensive review and statistical critique. , 2001, The Annals of thoracic surgery.

[20]  R. Heller,et al.  Accuracy of administrative data to assess comorbidity in patients with heart disease. an Australian perspective. , 2001, Journal of clinical epidemiology.

[21]  J. Tu,et al.  A Canadian comparison of data sources for coronary artery bypass surgery outcome "report cards". , 2000, American heart journal.

[22]  L I Iezzoni,et al.  Identification of in-hospital complications from claims data. Is it valid? , 2000, Medical care.

[23]  L I Iezzoni,et al.  Does clinical evidence support ICD-9-CM diagnosis coding of complications? , 2000, Medical care.

[24]  C. Vahl,et al.  The meaning of early mortality after CABG. , 1999, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[25]  J. Jollis,et al.  Pennsylvania's Focus on Heart Attack--grading the scorecard. , 1998, The New England journal of medicine.

[26]  L. Iezzoni,et al.  Predicting in-hospital deaths from coronary artery bypass graft surgery. Do different severity measures give different predictions? , 1998, Medical care.

[27]  C. Steiner,et al.  Comorbidity measures for use with administrative data. , 1998, Medical care.

[28]  L I Iezzoni,et al.  The risks of risk adjustment. , 1997, JAMA.

[29]  C D Naylor,et al.  Assessing the outcomes of coronary artery bypass graft surgery: how many risk factors are enough? Steering Committee of the Cardiac Care Network of Ontario. , 1997, Journal of the American College of Cardiology.

[30]  L. Iezzoni.  Assessing Quality Using Administrative Data , 1997, Annals of Internal Medicine.

[31]  C. Bombardier,et al.  Accuracy of administrative data for assessing outcomes after knee replacement surgery. , 1997, Journal of clinical epidemiology.

[32]  L I Iezzoni,et al.  How Much Are We Willing To Pay for Information about Quality of Care , 1997, Annals of Internal Medicine.

[33]  E. Hannan,et al.  Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough? , 1997, Health services research.

[34]  E L Hannan,et al.  Identification of preoperative variables needed for risk adjustment of short-term mortality after coronary artery bypass graft surgery. The Working Group Panel on the Cooperative CABG Database Project. , 1996, Journal of the American College of Cardiology.

[35]  J. Allison.  Risk adjustment for measuring health care outcomes , 1996 .

[36]  L. Iezzoni,et al.  Judging hospitals by severity-adjusted mortality rates: the case of CABG surgery. , 1996, Inquiry : a journal of medical care organization, provision and financing.

[37]  N. Wintfeld,et al.  Report cards on cardiac surgeons. Assessing New York State's approach. , 1995, The New England journal of medicine.

[38]  J. Daley.  Criteria by which to evaluate risk-adjusted outcomes programs in cardiac surgery. , 1994, The Annals of thoracic surgery.

[39]  E. Kuhn,et al.  Comparing hospitals that perform coronary artery bypass surgery: the effect of outcome measures and data sources. , 1994, American journal of public health.

[40]  L I Iezzoni,et al.  Identifying Complications of Care Using Administrative Data , 1994, Medical care.

[41]  L. Iezzoni.  Risk Adjustment for Measuring Healthcare Outcomes , 1994 .

[42]  J. Jollis,et al.  A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. Ischemic Heart Disease Patient Outcomes Research Team. , 1994, Journal of clinical epidemiology.

[43]  D. Mark,et al.  Bias in the coding of hospital discharge data and its implications for quality assessment. , 1994, Medical care.

[44]  E. DeLong,et al.  Discordance of Databases Designed for Claims Payment versus Clinical Information Systems: Implications for Outcomes Research , 1993, Annals of Internal Medicine.

[45]  P E Dans,et al.  Looking for Answers in All the Wrong Places , 1993, Annals of Internal Medicine.

[46]  J. Jollis,et al.  Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. , 1993, Journal of clinical epidemiology.

[47]  E L Hannan,et al.  Clinical Versus Administrative Data Bases for CABG Surgery: Does it Matter , 1992, Medical care.

[48]  R. Deyo,et al.  Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. , 1992, Journal of clinical epidemiology.

[49]  S. Jencks,et al.  Accuracy in recorded diagnoses. , 1992, JAMA.

[50]  E. Fisher,et al.  Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality? , 1992, JAMA.

[51]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .