Discrimination measures for survival outcomes: Connection between the AUC and the predictiveness curve

Finding out biomarkers and building risk scores to predict the occurrence of survival outcomes is a major concern of clinical epidemiology, and so is the evaluation of prognostic models. In this paper, we are concerned with the estimation of the time-dependent AUC--area under the receiver-operating curve--which naturally extends standard AUC to the setting of survival outcomes and enables to evaluate the discriminative power of prognostic models. We establish a simple and useful relation between the predictiveness curve and the time-dependent AUC--AUC(t). This relation confirms that the predictiveness curve is the key concept for evaluating calibration and discrimination of prognostic models. It also highlights that accurate estimates of the conditional absolute risk function should yield accurate estimates for AUC(t). From this observation, we derive several estimators for AUC(t) relying on distinct estimators of the conditional absolute risk function. An empirical study was conducted to compare our estimators with the existing ones and assess the effect of model misspecification--when estimating the conditional absolute risk function--on the AUC(t) estimation. We further illustrate the methodology on the Mayo PBC and the VA lung cancer data sets.

[1]  Laurence L. George,et al.  The Statistical Analysis of Failure Time Data , 2003, Technometrics.

[2]  O. Aalen A linear regression model for the analysis of life times. , 1989, Statistics in medicine.

[3]  Ying Huang,et al.  Evaluating the ROC performance of markers for future events , 2008, Lifetime data analysis.

[4]  Li-Hsing Shih,et al.  Variate generation for accelerated life and proportional hazards models with time dependent covariates , 1990 .

[5]  Vivian Viallon,et al.  How to evaluate the calibration of a disease risk prediction tool , 2007, Statistics in medicine.

[6]  E. Mammen,et al.  Comparing Nonparametric Versus Parametric Regression Fits , 1993 .

[7]  T. Lumley,et al.  Time‐Dependent ROC Curves for Censored Survival Data and a Diagnostic Marker , 2000, Biometrics.

[8]  D.,et al.  Regression Models and Life-Tables , 2022 .

[9]  P. Heagerty,et al.  Survival Model Predictive Accuracy and ROC Curves , 2005, Biometrics.

[10]  M. Schemper Predictive accuracy and explained variation , 2003, Statistics in medicine.

[11]  M. Akritas Nearest Neighbor Estimation of a Bivariate Distribution Under Random Censoring , 1994 .

[12]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data: Kalbfleisch/The Statistical , 2002 .

[13]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data , 1980 .

[14]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[15]  Lori E. Dodd,et al.  Partial AUC Estimation and Regression , 2003, Biometrics.

[16]  Yingye Zheng,et al.  Integrating the predictiveness of a marker with its performance as a classifier. , 2007, American journal of epidemiology.

[17]  A. Földes,et al.  STRONG UNIFORM CONSISTENCY FOR NONPARAMETRIC SURVIVAL CURVE ESTIMATORS FROM RANDOMLY CENSORED DATA , 1981 .

[18]  F. Harrell,et al.  Evaluating the yield of medical tests. , 1982, JAMA.

[19]  Mitchell H Gail,et al.  On criteria for evaluating models of absolute risk. , 2005, Biostatistics.

[20]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[21]  M. Gonen,et al.  Concordance probability and discriminatory power in proportional hazards regression , 2005 .

[22]  D. Cox,et al.  A General Definition of Residuals , 1968 .

[23]  R. Gill,et al.  History of applications of martingales in survival analysis. , 2010, 1003.0188.

[24]  Ziding Feng,et al.  Evaluating the Predictiveness of a Continuous Marker , 2007, Biometrics.

[25]  D. Hosmer,et al.  A review of goodness of fit statistics for use in the development of logistic regression models. , 1982, American journal of epidemiology.

[26]  Guoqing Diao,et al.  Estimation of time‐dependent area under the ROC curve for long‐term risk prediction , 2006, Statistics in medicine.

[27]  Margaret Pepe,et al.  Measures to Summarize and Compare the Predictive Capacity of Markers , 2009, The international journal of biostatistics.

[28]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.