Hazards of benchmarking complications with the National Trauma Data Bank: numerators in search of denominators.

BACKGROUND Complication rates after trauma may serve as important indicators of quality of care. Meaningful performance benchmarks for complication rates require reference standards from valid and reliable data. Selection of appropriate numerators and denominators is a major consideration for data validity in performance improvement and benchmarking. We examined the suitability of the National Trauma Data Bank (NTDB) as a reference for benchmarking trauma center complication rates. METHOD We selected the five most commonly reported complications in the NTDB v. 6.1 (pneumonia, urinary tract infection, acute respiratory distress syndrome, deep vein thrombosis, myocardial infarction). We compared rates for each complication using three different denominators defined by different populations at risk. A-all patients from all 700 reporting facilities as the denominator (n = 1,466,887); B-only patients from the 441 hospitals reporting at least one complication (n = 1,307,729); C-patients from hospitals reporting at least one occurrence of each specific complication, giving a unique denominator for each complication (n range = 869,675-1,167,384). We also looked at differences in hospital characteristics between complication reporters and nonreporters. RESULTS There was a 12.2% increase in the rate of each complication when patients from facilities not reporting any complications were excluded from the denominator. When rates were calculated using a unique denominator for each complication, rates increased 25% to 70%. The change from rate A to rate C produced a new rank order for the top five complications. When compared directly, rates B and C were also significantly different for all complications (all p < 0.01). Hospitals that reported complication information had significantly higher annual admissions and were more likely to be designated level I or II trauma centers and be university teaching hospitals. CONCLUSION There is great variability in complication data reported in the NTDB that may introduce bias and significantly influence rates of complications reported. This potential for bias creates a challenge for appropriately interpreting complication rates for hospital performance benchmarking. We recognize the value of large aggregated registries such as the NTDB as a valuable tool for benchmarking and performance improvement purposes. However, we strongly advocate the need for conscientious selection of numerators and denominators that serve as the basic foundation for research.

[1]  P. Rosenbaum,et al.  A spurious correlation between hospital mortality and complication rates: the importance of severity adjustment. , 1997, Medical care.

[2]  N. Mcswain,et al.  Accuracy of administrative and trauma registry databases. , 2001, The Journal of trauma.

[3]  D. Clark.  Comparing institutional trauma survival to a standard: current limitations and suggested alternatives. , 1999, The Journal of trauma.

[4]  Paying the price of excluding patients from a trauma registry. , 2006 .

[5]  M. Aaland,et al.  Trauma registry data validation: Essential for quality trauma care. , 2006, The Journal of trauma.

[6]  J. Fleishman,et al.  Dental service use among adults with human immunodeficiency virus infection. , 1997, Medical care.

[7]  B. Gabbe,et al.  Predictors of mortality, length of stay and discharge destination in blunt trauma , 2005, ANZ journal of surgery.

[8]  K. Shojania,et al.  The tension between needing to improve care and knowing how to do it. , 2007, The New England journal of medicine.

[9]  T. Osler,et al.  Beyond the major trauma outcome study: benchmarking performance using a national contemporary, population-based trauma registry. , 2001, The Journal of trauma.

[10]  Marlene R. Miller,et al.  Tracking progress in patient safety: an elusive target. , 2006, JAMA.

[11]  T. Osler,et al.  Impact of Changing the Statistical Methodology on Hospital and Surgeon Ranking: The Case of the New York State Cardiac Surgery Report Card , 2006, Medical care.

[12]  W. Henderson,et al.  Postoperative Mortality and Pulmonary Complication Rankings: How Well Do They Correlate at the Hospital Level? , 2003, Medical care.

[13]  R. Thomson,et al.  Using routine comparative data to assess the quality of health care: understanding and avoiding common pitfalls , 2003, Quality & safety in health care.

[14]  P. Pronovost,et al.  Can increased incidence of deep vein thrombosis (DVT) be used as a marker of quality of care in the absence of standardized screening? The potential effect of surveillance bias on reported DVT rates after trauma. , 2007, The Journal of trauma.

[15]  D. Ransohoff,et al.  Accuracy of administrative data in trauma. , 1999, Surgery.

[16]  L. Walter,et al.  Pitfalls of converting practice guidelines into quality measures: lessons learned from a VA performance measure. , 2004, JAMA.

[17]  D. Brailer,et al.  Comorbidity-adjusted complication risk: a new outcome quality measure. , 1996, Medical care.

[18]  Eugene H Blackstone,et al.  Monitoring surgical performance. , 2004, The Journal of thoracic and cardiovascular surgery.

[19]  C. Patterson.  Joint Commission on Accreditation of Healthcare Organizations , 1995, Infection Control &#x0026; Hospital Epidemiology.

[20]  L. Iezzoni.  Assessing Quality Using Administrative Data , 1997, Annals of Internal Medicine.

[21]  D. Asch,et al.  The unintended consequences of publicly reporting quality information. , 2005, JAMA.

[22]  W. Lamorte,et al.  Factors associated with mortality in trauma: re-evaluation of the TRISS method using the National Trauma Data Bank. , 2004, The Journal of trauma.

[23]  Patrick S Romano,et al.  Can Administrative Data Be Used to Compare Postoperative Complication Rates Across Hospitals? , 2002, Medical care.