Some prognostic models for traumatic brain injury were not valid.

OBJECTIVE Various prognostic models have been developed to predict outcome after traumatic brain injury (TBI). We aimed to determine the validity of six models that used baseline clinical and computed tomographic characteristics to predict mortality or unfavorable outcome at 6 months or later after severe or moderate TBI. STUDY DESIGN AND SETTING The validity was studied in two selected series of TBI patients enrolled in clinical trials (Tirilazad trials; n = 2,269; International Selfotel Trial; n = 409) and in two unselected series of patients consecutively admitted to participating centers (European Brain Injury Consortium [EBIC] survey; n = 796; Traumatic Coma Data Bank; n = 746). Validity was indicated by discriminative ability (AUC) and calibration (Hosmer-Lemeshow goodness-of-fit test). RESULTS The models varied in number of predictors (four to seven) and in development technique (two prediction trees and four logistic regression models). Discriminative ability varied widely (AUC: .61-.89), but calibration was poor for most models. Better discrimination was observed for logistic regression models compared with trees, and for models including more predictors. Further, discrimination was better when tested on unselected series that contained more heterogeneous populations. CONCLUSION Our findings emphasize the need for external validation of prognostic models. The satisfactory discrimination indicates that logistic regression models, developed on large samples, can be used for classifying TBI patients according to prognostic risk.

[1]  A. Benzer,et al.  Prediction of non-survival after trauma: Innsbruck Coma Scale , 1991, The Lancet.

[2]  R. Bullock,et al.  Failure of the competitive N-methyl-d-aspartate antagonist Selfotel (CGS 19755) in the treatment of severe head injury: results of two Phase III clinical trials , 1999 .

[3]  J M Wardlaw,et al.  Predicting survival using simple clinical variables: a case study in traumatic brain injury , 1999, Journal of neurology, neurosurgery, and psychiatry.

[4]  Daniel B. Mark,et al.  TUTORIAL IN BIOSTATISTICS MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS , 1996 .

[5]  Roderick J. A. Little.  Regression with Missing X's: A Review , 1992 .

[6]  J. Habbema,et al.  Regional differences in patient characteristics, case management, and outcomes in traumatic brain injury: experience from the tirilazad trials. , 2002, Journal of neurosurgery.

[7]  D. Titterington,et al.  Comparison of Discrimination Techniques Applied to a Complex Data Set of Head Injured Patients , 1981 .

[8]  E. Steyerberg,et al.  Predicting survival using simple clinical variables: a case study in traumatic brain injury , 2000, Journal of neurology, neurosurgery, and psychiatry.

[9]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[10]  B. Jennett,et al.  Assessment of coma and impaired consciousness. A practical scale. , 1974, Lancet.

[11]  J. C. van Houwelingen,et al.  Predictive value of statistical models , 1990 .

[12]  Jesse A. Berlin,et al.  Assessing the Generalizability of Prognostic Information , 1999 .

[13]  J. Passagia,et al.  Severe head injuries: An outcome prediction and survival analysis , 1996, Intensive Care Medicine.

[14]  Ewout W Steyerberg,et al.  Validation and updating of predictive logistic regression models: a study on sample size and shrinkage , 2004, Statistics in medicine.

[15]  J. Habbema,et al.  Systematic selection of prognostic features in patients with severe head injury. , 1980, Neurosurgery.

[16]  M. Radmacher,et al.  Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. , 2003, Journal of the National Cancer Institute.

[17]  Ewout W Steyerberg,et al.  Predicting outcome after traumatic brain injury: development and validation of a prognostic score based on admission characteristics. , 2005, Journal of neurotrauma.

[18]  B. Jennett,et al.  ASSESSMENT OF OUTCOME AFTER SEVERE BRAIN DAMAGE A Practical Scale , 1975, The Lancet.

[19]  J. Habbema,et al.  Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. , 2000, Statistics in medicine.

[20]  Vincent Corruble,et al.  Predicting recovery in patients suffering from traumatic brain injury by using admission variables and physiological data: a comparison between decision tree analysis and logistic regression. , 2002, Journal of neurosurgery.

[21]  J. Habbema,et al.  Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. , 2001, Journal of clinical epidemiology.

[22]  Frank E. Harrell,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[23]  D G Altman,et al.  What do we mean by validating a prognostic model? , 2000, Statistics in medicine.

[24]  G. Murray,et al.  Evaluation of designs for clinical trials of neuroprotective agents in head injury. European Brain Injury Consortium. , 1999, Journal of neurotrauma.

[25]  A. Maas,et al.  The European Brain Injury Consortium , 1997, Acta Neurochirurgica.

[26]  G C Sakellaropoulos,et al.  Development of a Bayesian Network for the Prognosis of Head Injuries using Graphical Model Selection Techniques , 1999, Methods of Information in Medicine.

[27]  A. Marmarou,et al.  Prediction tree for severely head-injured patients. , 1991, Journal of neurosurgery.

[28]  Foss Mv.  MANAGEMENT OF VIRUS HEPATITIS. , 1964 .

[29]  R G Grossman,et al.  The National Traumatic Coma Data Bank. Part 1: Design, purpose, goals, and results. , 1983, Journal of neurosurgery.

[30]  B. Walters,et al.  Part 1: Guidelines for the management of severe traumatic brain injury , 2000 .

[31]  S Greenland,et al.  A critical look at methods for handling missing covariates in epidemiologic regression analyses. , 1995, American journal of epidemiology.

[32]  Joanna M Wardlaw,et al.  Adding insult to injury: the prognostic value of early secondary insults for survival after traumatic brain injury , 1999, Journal of neurology, neurosurgery, and psychiatry.

[33]  Noriaki Aoki,et al.  Determinants of mortality in patients with severe blunt head injury. , 2002, Archives of surgery.

[34]  J. Habbema,et al.  Prognostic Modeling with Logistic Regression Analysis , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[35]  E W Steyerberg,et al.  Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. , 1999, Journal of clinical epidemiology.

[36]  M. Fylaktakis,et al.  Outcome in severely head injured patients with and without multiple trauma. , 1998, Journal of neurosurgical sciences.

[37]  S. C. Choi,et al.  Temporal profile of outcomes in severe head injury. , 1994, Journal of neurosurgery.

[38]  M. Dearden,et al.  The European Brain Injury Consortium Survey of Head Injuries , 1999, Acta Neurochirurgica.

[39]  B. Boszczyk,et al.  Predictors of outcome in head injury: proposal of a new scaling system. , 2002, The Journal of trauma.

[40]  Lennart Persson,et al.  A multicenter trial on the efficacy of using tirilazad mesylate in cases of head injury , 1998 .

[41]  Douglas G Altman,et al.  Developing a prognostic model in the presence of missing data: an ovarian cancer case study. , 2003, Journal of clinical epidemiology.

[42]  D. Ransohoff.  Rules of evidence for cancer molecular-marker discovery and validation , 2004, Nature Reviews Cancer.