Evidence for quality indicators to evaluate adult trauma care: A systematic review*

Objective:Multiple quality indicators are available to evaluate adult trauma care, but their characteristics and outcomes have not been systematically compared. We sought to systematically review the evidence about the reliability, validity, and implementation of quality indicators for evaluating trauma care. Data Sources:Search of MEDLINE, EMBASE, CINAHL, and The Cochrane Library up to January 14, 2009; the Gray Literature; select journals by hand; reference lists; and articles recommended by experts in the field. Study Selection:Studies were selected that evaluated the reliability, validity, or the impact of one or more quality indicators on the quality of care delivered to patients ≥18 yrs of age with a major traumatic injury. Data Extraction:Reviewers with methodologic and content expertise conducted data extraction independently. Data Synthesis:The literature search identified 6869 citations. Review of abstracts led to the retrieval of 538 full-text articles for assessment; 40 articles were selected for review. Of these, 20 (50%) articles were cohort studies and 13 (33%) articles were case series. Five articles used control groups, including three before and after case series, a case-control study, and a nonrandomized controlled trial. A total of 115 quality indicators in adult trauma care was identified, predominantly measures of hospital processes (62%) and outcomes (17%) of care. We did not identify any posthospital or secondary injury prevention quality indicators. Reliability was described for two quality indicators, content validity for 22 quality indicators, construct validity for eight quality indicators, and criterion validity for 46 quality indicators. A total of 58 quality indicators was implemented and evaluated in three studies. Eight quality indicators had supporting evidence for more than one measurement domain. A single quality indicator, peer review for preventable death, had both reliability and validity evidence. Conclusions:Although many quality indicators are available to measure the quality of trauma care, reliability evidence, validity evidence, and description of outcomes after implementation are limited.

[1]  S. Straus,et al.  Quality indicators for evaluating trauma care: a scoping review. , 2010, Archives of surgery.

[2]  H. Stelfox,et al.  The number, content, and quality of randomized controlled trials in the prevention and care of injuries. , 2008, The Journal of trauma.

[3]  G. Nardi,et al.  Are the ACSCOT filters associated with outcome? Examining morbidity and mortality in a European setting. , 2008, Injury.

[4]  Todd Dorman,et al.  The Reported Validity and Reliability of Methods for Evaluating Continuing Medical Education: A Systematic Review , 2008, Academic medicine : journal of the Association of American Medical Colleges.

[5]  Ramon Diaz-Arrastia,et al.  Intracranial pressure monitoring in brain-injured patients is associated with worsening of survival. , 2008, The Journal of trauma.

[6]  P. Cameron,et al.  Interpreting process indicators in trauma care: construct validity versus confounding by indication. , 2007, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[7]  M. Schiff,et al.  The identification of criteria to evaluate prehospital trauma care using the Delphi technique. , 2007, The Journal of trauma.

[8]  Francesca Valent,et al.  Are pre-hospital time and emergency department disposition time useful process indicators for trauma care in Italy? , 2007, Injury.

[9]  O. Bouamra,et al.  Outcome prediction in trauma. , 2006, Injury.

[10]  R. Maier,et al.  Patterns of Errors Contributing to Trauma Mortality: Lessons Learned From 2594 Deaths , 2006, Annals of surgery.

[11]  Edbert B. Hsu,et al.  Challenges in Systematic Reviews of Educational Intervention Studies , 2005, Annals of Internal Medicine.

[12]  T. Osler,et al.  Evaluating trauma center quality: does the choice of the severity-adjustment model make a difference? , 2005, The Journal of trauma.

[13]  G. Elwyn,et al.  Learning in practice Review of instruments for peer assessment of physicians , 2004 .

[14]  M. Al-Naami,et al.  Quality improvement data analysis of a mass casualty event. , 2003, Injury.

[15]  E. McGlynn.  Selecting Common Measures of Quality and System Performance , 2003, Medical care.

[16]  J. Debatin,et al.  A multidisciplinary quality management system for the early treatment of severely injured patients: implementation and results in two trauma centers , 2002, Intensive Care Medicine.

[17]  J. Nicholl,et al.  Determining the most effective level of TRISS-derived probability of survival for use as an audit filter. , 2002, Emergency medicine.

[18]  R. Karmy-Jones,et al.  Is delayed laparotomy for blunt abdominal trauma a valid quality improvement measure in the era of nonoperative management of abdominal injuries? , 2002, The Journal of trauma.

[19]  S. Cordner,et al.  Road traffic fatalities in Victoria, Australia and changes to the trauma care system , 2001, The British journal of surgery.

[20]  W. Chadbunchachai,et al.  The comparative study for quality of trauma treatment before and after the revision of trauma audit filter, Khon Kaen hospital 1998. , 2001, Journal of the Medical Association of Thailand = Chotmaihet thangphaet.

[21]  G. O’Keefe,et al.  Defining excess resource utilization and identifying associated factors for trauma victims. , 1999, The Journal of trauma.

[22]  K. Takayanagi,et al.  Preventable trauma deaths: evaluation by peer review and a guide for quality improvement. Emergency Medical Study Group for Quality. , 1998 .

[23]  W. Shoemaker,et al.  TRISS methodology in trauma: the need for alternatives , 1998, The British journal of surgery.

[24]  S. Cordner,et al.  Reproducibility of preventable death judgments and problem identification in 60 consecutive road trauma fatalities in Victoria, Australia. Consultative Committee on Road Traffic Fatalities in Victoria. , 1997, The Journal of trauma.

[25]  Alan D. Lopez,et al.  Mortality by cause for eight regions of the world: Global Burden of Disease Study , 1997, The Lancet.

[26]  W. Fallon,et al.  Benchmarking the quality-monitoring process: a comparison of outcomes analysis by trauma and injury severity score (TRISS) methodology with the peer-review process. , 1997, The Journal of trauma.

[27]  J. Hiatt,et al.  Continuous use of standard process audit filters has limited value in an established trauma system. , 1996, The Journal of trauma.

[28]  R. Maio,et al.  A study of preventable trauma mortality in rural Michigan. , 1996, The Journal of trauma.

[29]  S. Cordner,et al.  Evaluation of the medical management and preventability of death in 137 road traffic fatalities in Victoria, Australia: an overview. Consultative Committee on Road Traffic Fatalities in Victoria. , 1996, The Journal of trauma.

[30]  D. Gorman,et al.  Preventable deaths among major trauma patients in Mersey Region, North Wales and the Isle of Man. , 1996, Injury.

[31]  T. Esposito,et al.  Analysis of preventable trauma deaths and inappropriate trauma care in a rural state. , 1995, The Journal of trauma.

[32]  C. Wolferth,et al.  Analysis of American College of Surgeons trauma consultation program. , 1995, Archives of surgery.

[33]  W. Sacco,et al.  American College of Surgeons audit filters: associations with patient outcome and resource utilization. , 1995, The Journal of trauma.

[34]  N. Stocchetti,et al.  Trauma care in Italy: evidence of in-hospital preventable deaths. , 1994, The Journal of trauma.

[35]  R. Rutledge,et al.  American College of Surgeons trauma quality indicators: an analysis of outcome in a statewide trauma registry. , 1993, The Journal of trauma.

[36]  E. Mackenzie,et al.  Inter-rater reliability of preventable death judgments. The Preventable Death Study Group. , 1992, The Journal of trauma.

[37]  D. Hill,et al.  Evaluation of TRISS as a means of selecting trauma deaths for clinical peer review. , 1992, The Australian and New Zealand journal of surgery.

[38]  G. Rozycki,et al.  Results of a multi-institutional outcome assessment: results of a structured peer review of TRISS-designated unexpected outcomes. , 1992, The Journal of trauma.

[39]  I. Phair,et al.  Preventable deaths after head injury: a clinical audit of performance. , 1991, Injury.

[40]  P. Sharkey,et al.  Quality assurance for patients with head injuries admitted to a regional trauma unit. , 1990, The Journal of trauma.

[41]  F. Rivara,et al.  Validity of applying adult TRISS analysis to injured children. , 1990, The Journal of trauma.

[42]  D. Schriger,et al.  A comparison of implicit and explicit methods of process quality assurance for blunt trauma patients. , 1990, Annals of emergency medicine.

[43]  M. Rhodes,et al.  Cost effectiveness of trauma quality assurance audit filters. , 1990, The Journal of trauma.

[44]  D. Diamond,et al.  Acute subdural hematoma: morbidity and mortality related to timing of operative intervention. , 1990, The Journal of trauma.

[45]  R. Gamelli,et al.  Practical evaluation of trauma deaths. , 1989, The Journal of trauma.

[46]  R. Goris,et al.  Preventable trauma deaths in The Netherlands--a prospective multicenter study. , 1989, The Journal of trauma.

[47]  F. T. de Dombal,et al.  Retrospective study of 1000 deaths from injury in England and Wales , 1988, British medical journal.

[48]  J. G. West.  Validation of autopsy method for evaluating trauma care. , 1982, Archives of surgery.

[49]  W. Haddon,et al.  The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. , 1974, The Journal of trauma.

[50]  A. Donabedian.  Evaluating the quality of medical care. 1966. , 1966, The Milbank quarterly.

[51]  F. Brenneman,et al.  Preventable deaths from hemorrhage at a level I Canadian trauma center. , 2007, The Journal of trauma.

[52]  T. Karchmer,et al.  National nosocomial infection surveillance system: from benchmark to bedside in trauma patients. , 2006, The Journal of trauma.

[53]  T. Osler,et al.  Judging trauma center quality: does it depend on the choice of outcomes? , 2004, The Journal of trauma.

[54]  W. Chadbunchachai,et al.  Study on performance following Key Performance Indicators for trauma care: Khon Kaen Hospital 2000. , 2003, Journal of the Medical Association of Thailand = Chotmaihet thangphaet.

[55]  T R Miller,et al.  DATABOOK ON NONFATAL INJURY: INCIDENCE, COSTS, AND CONSEQUENCES , 1995 .

[56]  L. Fielding,et al.  Identification of preventable trauma deaths: confounded inquiries? , 1992, The Journal of trauma.

[57]  K. Lohr,et al.  The Definition of Quality and Approaches to Its Assessment. , 1981 .

[58]  José Maria Paganni,et al.  Evaluating the quality of medical care. , 1966, The Milbank Memorial Fund quarterly.