Evaluating incremental values from new predictors with net reclassification improvement in survival analysis

Developing individualized prediction rules for disease risk and prognosis has played a key role in modern medicine. When new genomic or biological markers become available to assist in risk prediction, it is essential to assess the improvement in clinical usefulness of the new markers over existing routine variables. Net reclassification improvement (NRI) has been proposed to assess improvement in risk reclassification in the context of comparing two risk models and the concept has been quickly adopted in medical journals (Pencina et al., Stat Med 27:157–172, 2008). We propose both nonparametric and semiparametric procedures for calculating NRI as a function of a future prediction time $$t$$ with a censored failure time outcome. The proposed methods accommodate covariate-dependent censoring, therefore providing more robust and sometimes more efficient procedures compared with the existing nonparametric-based estimators (Pencina et al., Stat Med 30:11–21, 2011; Uno et al., Comparing risk scoring systems beyond the roc paradigm in survival analysis, 2009). Simulation results indicate that the proposed procedures perform well in finite samples. We illustrate these procedures by evaluating a new risk model for predicting the onset of cardiovascular disease.

[1]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[2]  D. Lloyd‐Jones,et al.  Cardiovascular risk prediction: basic concepts, current status, and future directions. , 2010, Circulation.

[3]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[4]  Jisheng Cui,et al.  Overview of risk prediction models in cardiovascular disease research. , 2009, Annals of epidemiology.

[5]  L. J. Wei,et al.  The Robust Inference for the Cox Proportional Hazards Model , 1989 .

[6]  Allen J. Taylor,et al.  The Framingham Risk Score: an appraisal of its benefits and limitations. , 2007, The American heart hospital journal.

[7]  R. Gill,et al.  Cox's regression model for counting processes: a large sample study : (preprint) , 1982 .

[8]  M. Gail,et al.  Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. , 1989, Journal of the National Cancer Institute.

[9]  E. Topol,et al.  Prevalence of Conventional Risk Factors in Patients With Coronary Heart Disease , 2003 .

[10]  Hajime Uno,et al.  Calibrating parametric subject-specific risk estimation. , 2010, Biometrika.

[11]  Lu Tian,et al.  A unified inference procedure for a class of measures to assess improvement in risk prediction systems with survival data , 2013, Statistics in medicine.

[12]  Els Goetghebeur,et al.  Model evaluation based on the sampling distribution of estimated absolute prediction error , 2007 .

[13]  Tianxi Cai,et al.  Evaluating Prediction Rules for t-Year Survivors With Censored Regression Models , 2007 .

[14]  Nancy R Cook,et al.  The Effect of Including C-Reactive Protein in Cardiovascular Risk Prediction Models for Women , 2006, Annals of Internal Medicine.

[15]  Somnath Datta,et al.  The Kaplan–Meier Estimator as an Inverse-Probability-of-Censoring Weighted Average , 2001, The American statistician.

[16]  Margaret Pepe,et al.  Measures to Summarize and Compare the Predictive Capacity of Markers , 2009, The international journal of biostatistics.

[17]  W. Kannel,et al.  An investigation of coronary heart disease in families. The Framingham offspring study. , 1979, American journal of epidemiology.

[18]  Nils Lid Hjort,et al.  On inference in parametric survival data models , 1992 .

[19]  D. Levy,et al.  Prediction of coronary heart disease using risk factor categories. , 1998, Circulation.

[20]  Tianxi Cai,et al.  Comparing Risk Scoring Systems Beyond the ROC Paradigm in Survival Analysis , 2009 .

[21]  Tianxi Cai,et al.  Time-Dependent Predictive Values of Prognostic Biomarkers With Failure Time Outcome , 2008, Journal of the American Statistical Association.

[22]  D. Pollard Empirical Processes: Theory and Applications , 1990 .

[23]  Ewout W Steyerberg,et al.  Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers , 2011, Statistics in medicine.

[24]  Zhiliang Ying,et al.  Towards a general asymptotic theory for Cox model with staggered entry , 1997 .

[25]  Dorota M. Dabrowska,et al.  Smoothed Cox regression , 1997 .

[26]  N. Cook Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.