Dynamic thresholds and a summary ROC curve: Assessing prognostic accuracy of longitudinal markers

Cancer patients, chronic kidney disease patients, and subjects infected with HIV are routinely monitored over time using biomarkers that represent key health status indicators. Furthermore, biomarkers are frequently used to guide initiation of new treatments or to inform changes in intervention strategies. Since key medical decisions can be made on the basis of a longitudinal biomarker, it is important to evaluate the potential accuracy associated with longitudinal monitoring. To characterize the overall accuracy of a time‐dependent marker, we introduce a summary ROC curve that displays the overall sensitivity associated with a time‐dependent threshold that controls time‐varying specificity. The proposed statistical methods are similar to concepts considered in disease screening, yet our methods are novel in choosing a potentially time‐dependent threshold to define a positive test, and our methods allow time‐specific control of the false‐positive rate. The proposed summary ROC curve is a natural averaging of time‐dependent incident/dynamic ROC curves and therefore provides a single summary of net error rates that can be achieved in the longitudinal setting.

[1]  Lu Tian,et al.  A unified inference procedure for a class of measures to assess improvement in risk prediction systems with survival data , 2013, Statistics in medicine.

[2]  P. Heagerty,et al.  Exploration of Lagged Associations using Longitudinal Data , 2013, Biometrics.

[3]  P. Heagerty,et al.  Non-parametric estimation of a time-dependent predictive accuracy curve. , 2013, Biostatistics.

[4]  H. Putter,et al.  Dynamic Prediction in Clinical Survival Analysis , 2011 .

[5]  Dimitris Rizopoulos,et al.  Dynamic Predictions and Prospective Accuracy in Joint Models for Longitudinal and Time‐to‐Event Data , 2011, Biometrics.

[6]  W. Peacock,et al.  Serial changes in high‐sensitive troponin I predict outcome in patients with decompensated heart failure , 2011, European journal of heart failure.

[7]  P. Heagerty,et al.  Time‐Dependent Predictive Accuracy in the Presence of Competing Risks , 2010, Biometrics.

[8]  W. Koenig.  Integrating biomarkers: The new frontier? , 2010, Scandinavian journal of clinical and laboratory investigation. Supplementum.

[9]  Holly Janes,et al.  Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve. , 2009, Biometrika.

[10]  Xihong Lin,et al.  Semiparametric Modeling of Longitudinal Measurements and Time‐to‐Event Data–A Two‐Stage Regression Calibration Approach , 2008, Biometrics.

[11]  Tianxi Cai,et al.  The Performance of Risk Prediction Models , 2008, Biometrical journal. Biometrische Zeitschrift.

[12]  E Graf,et al.  Quantifying the Predictive Performance of Prognostic Models for Censored Survival Data with Time‐Dependent Covariates , 2008, Biometrics.

[13]  Sylvie Chevret,et al.  Assessing surrogacy from the joint modelling of multivariate longitudinal data and survival: Application to clinical trial data on chronic lymphocytic leukaemia , 2007, Statistics in medicine.

[14]  E. Mammen,et al.  A General Approach to the Predictability Issue in Survival Analysis with Applications , 2007 .

[15]  Jon A. Wellner,et al.  Empirical processes indexed by estimated functions , 2007, 0709.1013.

[16]  M. Schumacher,et al.  Consistent Estimation of the Expected Brier Score in General Survival Models with Right‐Censored Event Times , 2006, Biometrical journal. Biometrische Zeitschrift.

[17]  E. Elkin,et al.  Decision Curve Analysis: A Novel Method for Evaluating Prediction Models , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[18]  Yingye Zheng,et al.  Partly Conditional Survival Models for Longitudinal Data , 2005, Biometrics.

[19]  P. Heagerty,et al.  Survival Model Predictive Accuracy and ROC Curves , 2005, Biometrics.

[20]  Yingye Zheng,et al.  Semiparametric estimation of time-dependent ROC curves for longitudinal marker data. , 2004, Biostatistics.

[21]  Margaret Sullivan Pepe,et al.  The Analysis of Placement Values for Evaluating Discriminatory Measures , 2004, Biometrics.

[22]  Elizabeth R DeLong,et al.  ROC methodology within a monitoring framework , 2003, Statistics in medicine.

[23]  Kazuto Ito,et al.  Natural history of PSA increase with and without prostate cancer. , 2003, Urology.

[24]  Mary Lou Thompson,et al.  Assessing the diagnostic accuracy of a sequence of tests. , 2003, Biostatistics.

[25]  T. Lumley,et al.  Time‐Dependent ROC Curves for Censored Survival Data and a Diagnostic Marker , 2000, Biometrics.

[26]  M. Schemper,et al.  Predictive Accuracy and Explained Variation in Cox Regression , 2000, Biometrics.

[27]  S. Wieand,et al.  Comparison of diagnostic markers with repeated measurements: a non-parametric ROC curve approach. , 2000, Statistics in medicine.

[28]  L. Fisher,et al.  Time-dependent covariates in the Cox proportional-hazards regression model. , 1999, Annual review of public health.

[29]  S. Wieand,et al.  Analysis of repeated markers used to predict progression of cancer. , 1998, Statistics in medicine.

[30]  Nicholas P. Jewell,et al.  Marker processes in survival analysis , 1996, Lifetime data analysis.

[31]  ROC curves with multiple marker measurements. , 1995, Biometrics.

[32]  M. Akritas.  Nearest Neighbor Estimation of a Bivariate Distribution Under Random Censoring , 1994 .

[33]  Jens Perch Nielsen,et al.  A framework for consistent prediction rules based on markers , 1993 .

[34]  J. Phair,et al.  The Multicenter AIDS Cohort Study: rationale, organization, and selected characteristics of the participants. , 1987, American journal of epidemiology.

[35]  E. DeLong,et al.  Sensitivity and specificity of a monitoring test. , 1985, Biometrics.