Systematic review of predictive performance of injury severity scoring tools

Many injury severity scoring tools have been developed over the past few decades. These tools include the Injury Severity Score (ISS), New ISS (NISS), Trauma and Injury Severity Score (TRISS) and International Classification of Diseases (ICD)-based Injury Severity Score (ICISS). Although many studies have endeavored to determine the ability of these tools to predict the mortality of injured patients, their results have been inconsistent. We conducted a systematic review to summarize the predictive performances of these tools and explore the heterogeneity among studies. We defined a relevant article as any research article that reported the area under the Receiver Operating Characteristic curve as a measure of predictive performance. We conducted an online search using MEDLINE and Embase. We evaluated the quality of each relevant article using a quality assessment questionnaire consisting of 10 questions. The total number of positive answers was reported as the quality score of the study. Meta-analysis was not performed due to the heterogeneity among studies. We identified 64 relevant articles with 157 AUROCs of the tools. The median number of positive answers to the questionnaire was 5, ranging from 2 to 8. Less than half of the relevant studies reported the version of the Abbreviated Injury Scale (AIS) and/or ICD (37.5%). The heterogeneity among the studies could be observed in a broad distribution of crude mortality rates of study data, ranging from 1% to 38%. The NISS was mostly reported to perform better than the ISS when predicting the mortality of blunt trauma patients. The relative performance of the ICSS against the AIS-based tools was inconclusive because of the scarcity of studies. The performance of the ICISS appeared to be unstable because the performance could be altered by the type of formula and survival risk ratios used. In conclusion, high-quality studies were limited. The NISS might perform better in the mortality prediction of blunt injuries than the ISS. Additional studies are required to standardize the derivation of the ICISS and determine the relative performance of the ICISS against the AIS-based tools.

[1]  A. Hill,et al.  Venous glucose and arterial lactate as biochemical predictors of mortality in clinically severely injured trauma patients--a comparison with ISS and TRISS. , 2009, Injury.

[2]  G. Leung,et al.  Injury Severity Score (ISS) vs. ICD-derived Injury Severity Score (ICISS) in a patient population treated in a designated Hong Kong trauma centre , 2008, McGill journal of medicine : MJM : an international forum for the advancement of medical sciences by students.

[3]  Andrew Hunter,et al.  Application of neural networks and sensitivity analysis to improved prediction of trauma survival , 2000, Comput. Methods Programs Biomed..

[4]  E. Hannan,et al.  Predicting trauma inpatient mortality in an administrative database: an investigation of survival risk ratios using New York data. , 2007, The Journal of trauma.

[5]  Charles Mock,et al.  A comparison of the abilities of nine scoring algorithms in predicting mortality. , 2002, The Journal of trauma.

[6]  T. Osler,et al.  A Trauma Mortality Prediction Model Based on the Anatomic Injury Scale , 2008, Annals of surgery.

[7]  E. Mackenzie,et al.  Comparison of alternative methods for assessing injury severity based on anatomic descriptors. , 1999, The Journal of trauma.

[8]  I. Ercan,et al.  Should the New Injury Severity Score replace the Injury Severity Score in the Trauma and Injury Severity Score? , 2008, Ulusal travma ve acil cerrahi dergisi = Turkish journal of trauma & emergency surgery : TJTES.

[9]  Lynne Moore,et al.  The New Injury Severity Score: a more accurate predictor of in-hospital mortality than the Injury Severity Score. , 2004, The Journal of trauma.

[10]  S. Jamulitrat,et al.  A comparison of mortality predictive abilities between NISS and ISS in trauma patients. , 2001, Journal of the Medical Association of Thailand = Chotmaihet thangphaet.

[11]  T. K. Hunt,et al.  Trauma severity scoring to predict mortality , 2005, World Journal of Surgery.

[12]  L. Napolitano,et al.  Predictors of mortality in adult trauma patients: the physiologic trauma score is equivalent to the Trauma and Injury Severity Score. , 2002, Journal of the American College of Surgeons.

[13]  J. Savino,et al.  An artificial neural network as a model for prediction of survival in trauma patients: validation for a regional trauma area. , 1998, The Journal of trauma.

[14]  T Gennarelli,et al.  A new characterization of injury severity. , 1990, The Journal of trauma.

[15]  T. Osler,et al.  TMPM–ICD9: A Trauma Mortality Prediction Model Based on ICD-9-CM Codes , 2009, Annals of surgery.

[16]  M. Dennis,et al.  Systematic Review of Prognostic Models in Patients with Acute Stroke , 2001, Cerebrovascular Diseases.

[17]  Hilla Peretz,et al.  Ju n 20 03 Schrödinger ’ s Cat : The rules of engagement , 2003 .

[18]  T. Osler,et al.  Improving the TRISS methodology by restructuring age categories and adding comorbidities. , 2004, The Journal of trauma.

[19]  T. Osler,et al.  Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictors of outcome in injury: analysis of 44,032 patients. , 1997, The Journal of trauma.

[20]  Wenbin Wang,et al.  A Comparison Study of the Injury Score Models , 2006, European Journal of Trauma.

[21]  Mao Zhang,et al.  Comparison of the new injury severity score and the injury severity score in multiple trauma patients. , 2008, Chinese journal of traumatology = Zhonghua chuang shang za zhi.

[22]  C Cryer,et al.  Improving the predictive ability of the ICD-based Injury Severity Score , 2008, Injury Prevention.

[23]  P. Giannoudis,et al.  Which AIS based scoring system is the best predictor of outcome in orthopaedic blunt trauma patients? , 2006, The Journal of trauma.

[24]  D. Madigan,et al.  Bayesian logistic injury severity score: a method for predicting mortality using international classification of disease-9 codes. , 2008, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[25]  Yoon Kim,et al.  Validation of the International Classification of Diseases 10th Edition-based Injury Severity Score (ICISS). , 2000, The Journal of trauma.

[26]  W. Haddon,et al.  The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. , 1974, The Journal of trauma.

[27]  A. Nathens,et al.  Ranking of trauma center performance: the bare essentials. , 2008, The Journal of trauma.

[28]  T Wentzel-Larsen,et al.  Trauma patients in the intensive care unit: short‐ and long‐term survival and predictors of 30‐day mortality , 2007, Acta anaesthesiologica Scandinavica.

[29]  F P Rivara,et al.  Harborview assessment for risk of mortality: an improved measure of injury severity on the basis of ICD-9-CM. , 2000, The Journal of trauma.

[30]  B. Gabbe,et al.  Predictors of mortality, length of stay and discharge destination in blunt trauma , 2005, ANZ journal of surgery.

[31]  T. Osler,et al.  A fresh set of survival risk ratios derived from incidents in the National Trauma Data Bank from which the ICISS may be calculated. , 2003, The Journal of trauma.

[32]  K. J. Rhee,et al.  APACHE II scoring in the injured patient. , 1990, Critical care medicine.

[33]  Adina Zeki Al Hazzouri,et al.  The injury severity score or the new injury severity score for predicting mortality, intensive care unit admission and length of hospital stay: experience from a university hospital in a developing country. , 2008, Injury.

[34]  T. Osler,et al.  Injury severity scoring: perspectives in development and future directions. , 1993, American journal of surgery.

[35]  T. Osler,et al.  Trauma registry injury coding is superfluous: a comparison of outcome prediction based on trauma registry International Classification of Diseases-Ninth Revision (ICD-9) and hospital information system ICD-9 codes. , 1997, The Journal of trauma.

[36]  John D Langley,et al.  Comparing measures of injury severity for use with large databases. , 2002, The Journal of trauma.

[37]  P. Lane,et al.  An evaluation of Ontario trauma outcomes and the development of regional norms for Trauma and Injury Severity Score (TRISS) analysis. , 1996, The Journal of trauma.

[38]  B. Eftekhar,et al.  Comparing logistic models based on modified GCS motor component with other prognostic tools in prediction of mortality: results of study in 7226 trauma patients. , 2005, Injury.

[39]  E. Hannan,et al.  Predictors of mortality in adult patients with blunt injuries in New York State: a comparison of the Trauma and Injury Severity Score (TRISS) and the International Classification of Disease, Ninth Revision-based Injury Severity Score (ICISS). , 1999, The Journal of trauma.

[40]  J. Miquel,et al.  Epidemiologic aspects and results of applying the TRISS methodology in a Spanish trauma intensive care unit (TICU) , 1995, Intensive Care Medicine.

[41]  D A Redelmeier,et al.  Measuring injury severity: time for a change? , 1998, The Journal of trauma.

[42]  O. Bouamra,et al.  Outcome prediction in trauma. , 2006, Injury.

[43]  L. Zun,et al.  Comparison of the New Injury Severity Score and the Injury Severity Score. , 2004, The Journal of trauma.

[44]  O. Kobusingye,et al.  A Comparison of the Kampala Trauma Score (KTS) with the Revised Trauma Score (RTS), Injury Severity Score (ISS) and the TRISS Method in a Ugandan Trauma Registry , 2003, European Journal of Trauma.

[45]  J. Vogel.  A Trauma Mortality Prediction Model Based on the Anatomic Injury Scale , 2009 .

[46]  T. Osler,et al.  The ICD-9-based illness severity score: a new model that outperforms both DRG and APR-DRG as predictors of survival and resource utilization. , 1998, The Journal of trauma.

[47]  Ian Roberts,et al.  Systematic review of prognostic models in traumatic brain injury , 2006, BMC Medical Informatics Decis. Mak..

[48]  Rating the severity of tissue damage. I. The abbreviated scale. , 1971, JAMA.

[49]  T. Osler,et al.  Independently derived survival risk ratios yield better estimates of survival than traditional survival risk ratios when using the ICISS. , 2003, The Journal of trauma.

[50]  E. Hannan,et al.  A comparison among the abilities of various injury severity measures to predict mortality with and without accompanying physiologic information. , 2005, The Journal of trauma.

[51]  S P Baker,et al.  A modification of the injury severity score that both improves accuracy and simplifies scoring. , 1997, The Journal of trauma.

[52]  L. Leenen,et al.  Base deficit-based predictive modeling of outcome in trauma patients admitted to intensive care units in Dutch trauma centers. , 2007, The Journal of trauma.

[53]  P. Giannoudis,et al.  A comparison of European Trauma Registries. The first report from the EuroTARN Group. , 2007, Resuscitation.

[54]  T. Osler,et al.  Expert consensus vs empirical estimation of injury severity: effect on quality measurement in trauma. , 2009, Archives of surgery.

[55]  M. Moini,et al.  Application of "Trauma and Injury Severity Score" and "A Severity Characterization of Trauma" score to trauma patients in a setting different from "Major Trauma Outcome Study". , 2007, Archives of Iranian medicine.

[56]  T. Osler,et al.  Incorporating recent advances to make the TRISS approach universally available. , 2006, The Journal of trauma.

[57]  W. Lamorte,et al.  Factors associated with mortality in trauma: re-evaluation of the TRISS method using the National Trauma Data Bank. , 2004, The Journal of trauma.

[58]  R. Lefering,et al.  Trauma score systems: Cologne Validation Study. , 1997, The Journal of trauma.

[59]  T. Osler,et al.  ICISS: an international classification of disease-9 based injury severity score. , 1996, The Journal of trauma.

[60]  G. Pengo,et al.  [Prognostic systems in intensive care: TRISS, SAPS II, APACHE III]. , 2001, Minerva anestesiologica.

[61]  S. Fakhry,et al.  Injury severity grading in trauma patients: a simplified technique based upon ICD-9 coding. , 1993, The Journal of trauma.

[62]  T. Scalea,et al.  Prediction of outcomes in trauma: anatomic or physiologic parameters? , 2005, Journal of the American College of Surgeons.

[63]  W J Sacco,et al.  Progress toward a new injury severity characterization: severity profiles. , 1988, Computers in biology and medicine.

[64]  T Gennarelli,et al.  Progress in characterizing anatomic injury. , 1990, The Journal of trauma.

[65]  P. Metnitz,et al.  Improving risk adjustment in critically ill trauma patients: the TRISS-SAPS Score. , 2004, The Journal of trauma.

[66]  A. V. van Vugt,et al.  Comparison of current injury scales for survival chance estimation: an evaluation comparing the predictive performance of the ISS, NISS, and AP scores in a Dutch local trauma registration. , 2005, The Journal of trauma.

[67]  M. Émond,et al.  Consensus or data-derived anatomic injury severity scoring? , 2008, The Journal of trauma.

[68]  Y. Graaf,et al.  The Hospital Trauma Index , 2004, European Journal of Trauma.

[69]  P. Giannoudis,et al.  Predicting outcome after multiple trauma: which scoring system? , 2004, Injury.

[70]  Holly Janes,et al.  Practice of Epidemiology Adjusting for Covariates in Studies of Diagnostic, Screening, or Prognostic Markers: an Old Concept in a New Setting , 2022 .

[71]  T. Osler,et al.  Injury Severity Scoring , 1999 .

[72]  T. Coats,et al.  Comparison of artificial intelligence techniques with UKTRISS for estimating probability of survival after trauma. UK Trauma and Injury Severity Score. , 2001, The Journal of trauma.

[73]  D. Wong,et al.  A comparison of the Acute Physiology and Chronic Health Evaluation (APACHE) II score and the Trauma-Injury Severity Score (TRISS) for outcome assessment in intensive care unit trauma patients. , 1996, Critical care medicine.

[74]  A. Turgeon,et al.  The Trauma Risk Adjustment Model: A New Model for Evaluating Trauma Care , 2009, Annals of surgery.

[75]  Jill Hayden,et al.  Evaluation of the Quality of Prognosis Studies in Systematic Reviews , 2006, Annals of Internal Medicine.

[76]  P. Bossuyt,et al.  BMC Medical Research Methodology , 2002 .

[77]  M. Nijsten,et al.  Emergency trauma score: An instrument for early estimation of trauma severity* , 2009, Critical care medicine.

[78]  T. Osler,et al.  The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring. , 2003, The Journal of trauma.

[79]  F. Lewis,et al.  Prediction of outcome in intensive care unit trauma patients: a multicenter study of Acute Physiology and Chronic Health Evaluation (APACHE), Trauma and Injury Severity Score (TRISS), and a 24-hour intensive care unit (ICU) point system. , 1999, The Journal of trauma.

[80]  M. Kaufmann,et al.  Assessment of three different mortality prediction models in four well‐defined critical care patient groups at two points in time: a prospective cohort study , 2007, European journal of anaesthesiology.

[81]  R. Lefering.  Trauma Score Systems for Quality Assessment , 2002, European Journal of Trauma.

[82]  I. Stiell,et al.  Initial emergency department trauma scores from the OPALS study: the case for the motor score in blunt trauma. , 2004, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[83]  A. V. van Vugt,et al.  Predicting Survival after Trauma: a Comparison of TRISS and ASCOT in the Netherlands , 2002, European Journal of Trauma.

[84]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.