Comparison of mortality risk adjustment using a clinical data algorithm (American College of Surgeons National Surgical Quality Improvement Program) and an administrative data algorithm (Solucient) at the case level within a single institution.

BACKGROUND There is great interest in efficiently evaluating health care quality, but there is controversy over the use of administrative versus clinical data methods. We sought to compare actual mortality with risk-adjusted expected mortality in a sample population calculated by two different methods; one based on preexisting administrative records and one based on chart reviews. STUDY DESIGN We examined a sample of patients (n = 1,234) undergoing surgical procedures at an academic teaching hospital during 1 year. The first risk-adjustment method was that used by the National Surgical Quality Improvement Program, which is based on dedicated medical record review. The second method was that used by Solucient, LLC, which is based on preexisting administrative records. RESULTS The ratio of observed to expected mortality for this population set was higher using the National Surgical Quality Improvement Program algorithm (1.1; 95% CI, 0.8 to 1.5) than using the Solucient algorithm (0.9; 95% CI, 0.6 to 1.2) but neither estimate was notably different from 1.0. Similarly, when observed to expected mortality ratios were calculated separately for each quartile of mortality, there were no marked differences within quartiles, although minor differences with potential importance were noted. Fit was comparable by age categories, gender, and American Society of Anesthesiologists' categories. A number of actual deaths had higher predicted mortality scores using the Solucient algorithm. CONCLUSIONS Risk-adjusted mortality estimates were comparable using administrative or clinical data. Minor performance differences might still have implications. Because of the potential lower cost of using administrative data, this type of algorithm can be an efficient alternative and should continue to be investigated.

[1]  J. Birkmeyer,et al.  Hospital Volume and Surgical Mortality in the United States , 2002 .

[2]  Kwan Hur,et al.  Identifying patient preoperative risk factors and postoperative adverse events in administrative databases: results from the Department of Veterans Affairs National Surgical Quality Improvement Program. , 2002, Journal of the American College of Surgeons.

[3]  Mary S Vaughan-Sarrazin,et al.  Cardiac revascularization in specialty and general hospitals. , 2005, The New England journal of medicine.

[4]  P. Romano,et al.  Can Administrative Data Be Used to Ascertain Clinically Significant Postoperative Complications? , 2002, American journal of medical quality : the official journal of the American College of Medical Quality.

[5]  W. Weintraub,et al.  Can cardiovascular clinical characteristics be identified and outcome models be developed from an in-patient claims database? , 1999, The American journal of cardiology.

[6]  J Gibbs,et al.  The National Veterans Administration Surgical Risk Study: risk adjustment for the comparative assessment of the quality of surgical care. , 1995, Journal of the American College of Surgeons.

[7]  E. DeLong,et al.  Discordance of Databases Designed for Claims Payment versus Clinical Information Systems: Implications for Outcomes Research , 1993, Annals of Internal Medicine.

[8]  J. Alexander,et al.  Structural versus Outcomes Measures in Hospitals: A Comparison of Joint Commission and Medicare Outcomes Scores in Hospitals , 2002, Quality management in health care.

[9]  David J. Ballard.  Cardiac revascularization in specialty and general hospitals. , 2005 .

[10]  L I Iezzoni,et al.  Use of administrative data to find substandard care: validation of the complications screening program. , 2000, Medical care.

[11]  Saul N Weingart,et al.  Discrepancies between explicit and implicit review: physician and nurse assessments of complications and quality. , 2002, Health services research.

[12]  Jeffrey A. Alexander,et al.  Measuring Comparative Hospital Performance , 2002, Journal of healthcare management / American College of Healthcare Executives.

[13]  Patrick S Romano,et al.  Can Administrative Data Be Used to Compare Postoperative Complication Rates Across Hospitals? , 2002, Medical care.

[14]  J. Spinelli,et al.  Co-morbidity data in outcomes research: are clinical data derived from administrative databases a reliable alternative to chart review? , 2000, Journal of clinical epidemiology.

[15]  S. Leatherman,et al.  Using Claims Data for Epidemiologic Research: The Concordance of Claims-Based Criteria With the Medical Record and Patient Survey for Identifying a Hypertensive Population , 1993, Medical care.

[16]  Christianna S. Williams,et al.  Risk adjustment for older hospitalized persons: a comparison of two methods of data collection for the Charlson index. , 2001, Journal of clinical epidemiology.

[17]  D McLerran,et al.  Using administrative data to describe casemix: a comparison with the medical record. , 1994, Journal of clinical epidemiology.

[18]  J. Alexander,et al.  Is Anybody Managing the Store? National Trends in Hospital Performance , 2006, Journal of healthcare management / American College of Healthcare Executives.

[19]  D. Hoaglin,et al.  Enhancement of claims data to improve risk adjustment of hospital mortality. , 2007, JAMA.

[20]  F. Grover,et al.  Risk adjustment of the postoperative mortality rate for the comparative assessment of the quality of surgical care: results of the National Veterans Affairs Surgical Risk Study. , 1997, Journal of the American College of Surgeons.

[21]  W. Barlow,et al.  In Search of the Perfect Comorbidity Measure for Use With Administrative Claims Data: Does It Exist? , 2006, Medical care.

[22]  C. Klabunde,et al.  Data Sources for Measuring Comorbidity: A Comparison of Hospital Records and Medicare Claims for Cancer Patients , 2006, Medical care.

[23]  B. Griffin,et al.  Thromboprophylaxis in medically ill patients at risk for venous thromboembolism. , 2006, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[24]  L I Iezzoni,et al.  Does clinical evidence support ICD-9-CM diagnosis coding of complications? , 2000, Medical care.

[25]  P. McCollam,et al.  Cost and effectiveness of glycoprotein IIb/IIIa-receptor inhibitors in patients with acute myocardial infarction undergoing percutaneous coronary intervention. , 2003, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[26]  E. Hannan,et al.  Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough? , 1997, Health services research.

[27]  C. Guse,et al.  Severity of Illness Measures Derived From the Uniform Clinical Data Set (UCDSS) , 1994, Medical care.

[28]  Nancy J. Petersen,et al.  Case-Mix Adjustment Using Administrative Databases: A Paradigm to Guide Future Research , 1997, Medical care research and review : MCRR.

[29]  A. Atherly,et al.  Evaluating alternative risk-adjustment strategies for surgery. , 2004, American journal of surgery.

[30]  D. Carlisle,et al.  Administrative Versus Clinical Data for Coronary Artery Bypass Graft Surgery Report Cards: The View From California , 2006, Medical care.

[31]  Sunil Sinha,et al.  Hospital Boards and Quality Dashboards , 2006 .

[32]  Michael L. Johnson,et al.  Mortality After Noncardiac Surgery: Prediction From Administrative Versus Clinical Data , 2005, Medical care.

[33]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[34]  Harlan M Krumholz,et al.  An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With Heart Failure , 2006, Circulation.

[35]  M Pine,et al.  Predictions of Hospital Mortality Rates: A Comparison of Data Sources , 1997, Annals of Internal Medicine.

[36]  F. Grover,et al.  The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. , 1998, Annals of surgery.

[37]  G. Coffman,et al.  Predicting In-Hospital Mortality: A Comparison of Severity Measurement Approaches , 1992, Medical care.

[38]  L I Iezzoni,et al.  Identification of in-hospital complications from claims data. Is it valid? , 2000, Medical care.

[39]  C. Newschaffer,et al.  Comorbidity measurement in elderly female breast cancer patients with administrative and medical records data. , 1997, Journal of clinical epidemiology.

[40]  L I Iezzoni,et al.  Predicting Who Dies Depends on How Severity Is Measured: Implications for Evaluating Patient Outcomes , 1995, Annals of Internal Medicine.

[41]  D. Nordenberg,et al.  Epidemiology of Reye’s Syndrome, United States, 1991–1994: Comparison of CDC Surveillance and Hospital Admission Data , 2000, Neuroepidemiology.

[42]  C. Bombardier,et al.  Accuracy of administrative data for assessing outcomes after knee replacement surgery. , 1997, Journal of clinical epidemiology.

[43]  R. Holman,et al.  Kawasaki syndrome hospitalizations and associated costs in the United States. , 2003, Public health reports.

[44]  L I Iezzoni,et al.  Does the Complications Screening Program flag cases with process of care problems? Using explicit criteria to judge processes. , 1999, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[45]  E L Hannan,et al.  Clinical Versus Administrative Data Bases for CABG Surgery: Does it Matter , 1992, Medical care.

[46]  S. Normand,et al.  An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With an Acute Myocardial Infarction , 2006, Circulation.