Assessment of survival prediction models based on microarray data

MOTIVATION In the process of developing risk prediction models, various steps of model building and model selection are involved. If this process is not adequately controlled, overfitting may result in serious overoptimism leading to potentially erroneous conclusions. METHODS For right censored time-to-event data, we estimate the prediction error for assessing the performance of a risk prediction model (Gerds and Schumacher, 2006; Graf et al., 1999). Furthermore, resampling methods are used to detect overfitting and resulting overoptimism and to adjust the estimates of prediction error (Gerds and Schumacher, 2007). RESULTS We show how and to what extent the methodology can be used in situations characterized by a large number of potential predictor variables where overfitting may be expected to be overwhelming. This is illustrated by estimating the prediction error of some recently proposed techniques for fitting a multivariate Cox regression model applied to the data of a prognostic study in patients with diffuse large-B-cell lymphoma (DLBCL). AVAILABILITY Resampling-based estimation of prediction error curves is implemented in an R package called pec available from the authors.

[1]  David R. Cox,et al.  Regression models and life tables (with discussion , 1972 .

[2]  Richard Simon,et al.  Roadmap for developing and validating therapeutically relevant genomic classifiers. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[3]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[4]  E Graf,et al.  Assessment and comparison of prognostic classification schemes for survival data. , 1999, Statistics in medicine.

[5]  L. V. van't Veer,et al.  Cross‐validated Cox regression on microarray gene expression data , 2006, Statistics in medicine.

[6]  Jiang Gui,et al.  Penalized Cox regression analysis in the high-dimensional and low-sample size settings, with applications to microarray gene expression data , 2005, Bioinform..

[7]  D. Cox.  Regression Models and Life-Tables , 1972 .

[8]  T. Lumley,et al.  Time‐Dependent ROC Curves for Censored Survival Data and a Diagnostic Marker , 2000, Biometrics.

[9]  M. Segal.  Microarray gene expression data with linked survival phenotypes: diffuse large-B-cell lymphoma revisited. , 2006, Biostatistics.

[10]  Meland,et al.  THE USE OF MOLECULAR PROFILING TO PREDICT SURVIVAL AFTER CHEMOTHERAPY FOR DIFFUSE LARGE-B-CELL LYMPHOMA , 2002 .

[11]  Yang Jing.  L1 Regularization Path Algorithm for Generalized Linear Models , 2008 .

[12]  M. Schumacher,et al.  Consistent Estimation of the Expected Brier Score in General Survival Models with Right‐Censored Event Times , 2006, Biometrical journal. Biometrische Zeitschrift.

[13]  M. Radmacher,et al.  Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. , 2003, Journal of the National Cancer Institute.

[14]  Richard Simon,et al.  Explained Residual Variation, Explained Risk, and Goodness of Fit , 1991 .

[15]  Thomas A. Gerds,et al.  On functional misspecification of covariates in the Cox regression model , 2001 .

[16]  Wenjiang J. Fu,et al.  Estimating misclassification error with small samples via bootstrap cross-validation , 2005, Bioinform..

[17]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[18]  Jiang Gui,et al.  Partial Cox regression analysis for high-dimensional microarray gene expression data , 2004, ISMB/ECCB.

[19]  Michael Kattan,et al.  Statistical prediction models, artificial neural networks, and the sophism "I am a patient, not a statistic". , 2002, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[20]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[21]  P. Heagerty,et al.  Survival Model Predictive Accuracy and ROC Curves , 2005, Biometrics.

[22]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[23]  Annette M. Molinaro,et al.  Prediction error estimation: a comparison of resampling methods , 2005, Bioinform..

[24]  James M. Robins,et al.  Unified Methods for Censored Longitudinal Data and Causality , 2003 .

[25]  Thomas A Gerds,et al.  Efron‐Type Measures of Prediction Error for Survival Analysis , 2007, Biometrics.

[26]  J. Klein,et al.  Statistical Models Based On Counting Process , 1994 .

[27]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[28]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[29]  M. Schumacher,et al.  A Comparison of Nonparametric Error Rate Estimation Methods in Classification Problems , 2004 .

[30]  P. Garthwaite.  An Interpretation of Partial Least Squares , 1994 .

[31]  Edward R. Dougherty,et al.  Is cross-validation valid for small-sample microarray classification? , 2004, Bioinform..

[32]  D. Machin,et al.  Prognostic Factor Studies , 2005 .