Asymptotic and Finite Sample Behavior of Net Reclassification Indices

The Net Reclassification Index (NRI) introduced by Pencina and colleagues [1, 2] is designed to quantify the prediction increment provided by a new biomarker. It has become popular for evaluating and selecting novel markers. The published variance formulae for NRI statistics do not account for the fact that risks are estimated based on risk models fit to data, and thus are not valid in practice when estimated risks are used [3]. Kerr and colleagues [4] showed that the confidence intervals constructed based on a bootstrap estimate of the variance and Normal approximation had the best performance among various methods they examined, including the one based on bootstrap quantiles. This paper establishes asymptotic Normality of NRI statistics when true risks are unknown and are estimated. Our results provide theoretical support for constructing confidence intervals for NRI statistics based on a Normal approximation. We also derive explicit variance formulae for NRI statistics that are calculated based on estimated risks. In addition, we examine finite sample distributional behavior of NRI statistics in a simulation study. These results provide some guidance on the sample size required for adopting a Normal approximation for NRI inference in practice. Keyworks: Net reclassification index; incremental value; asymptotic Normality; sampling distribution; biomarker; risk models 1 Hosted by The Berkeley Electronic Press

[1]  Ewout W Steyerberg,et al.  Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers , 2011, Statistics in medicine.

[2]  Margaret S Pepe,et al.  Gauging the performance of SNPs, biomarkers, and clinical factors for predicting risk of breast cancer. , 2008, Journal of the National Cancer Institute.

[3]  Margaret S Pepe,et al.  Problems with risk reclassification methods for evaluating prediction models. , 2011, American journal of epidemiology.

[4]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[5]  P. Greenland,et al.  Coronary artery calcium score and risk classification for coronary heart disease prediction. , 2010, JAMA.

[6]  M H Gail,et al.  Validating and improving models for projecting the absolute risk of breast cancer. , 2001, Journal of the National Cancer Institute.

[7]  Kathleen F. Kerr,et al.  Testing for improvement in prediction model performance , 2013, Statistics in medicine.

[8]  M. Pencina,et al.  Interpreting incremental value of markers added to risk prediction models. , 2012, American journal of epidemiology.

[9]  M. Gail.  Value of adding single-nucleotide polymorphism genotypes to a breast cancer risk model. , 2009, Journal of the National Cancer Institute.

[10]  M. Gail.  Discriminatory accuracy from single-nucleotide polymorphisms in models to predict breast cancer risk. , 2008, Journal of the National Cancer Institute.

[11]  M. Thun,et al.  Performance of Common Genetic Variants in Breast-cancer Risk Models , 2022 .

[12]  Jinbo Chen,et al.  Projecting absolute invasive breast cancer risk in white women with a model that includes mammographic density. , 2006, Journal of the National Cancer Institute.

[13]  Thomas Lumley,et al.  American Journal of Epidemiology Practice of Epidemiology Evaluating the Incremental Value of New Biomarkers with Integrated Discrimination Improvement , 2022 .

[14]  Nancy R Cook,et al.  Advances in Measuring the Effect of Individual Predictors of Cardiovascular Risk: The Role of Reclassification Measures , 2009, Annals of Internal Medicine.

[15]  D. Levy,et al.  Prediction of coronary heart disease using risk factor categories. , 1998, Circulation.

[16]  Karla Kerlikowske,et al.  Prospective breast cancer risk prediction model for women undergoing screening mammography. , 2006, Journal of the National Cancer Institute.

[17]  N. Cook.  Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.

[18]  M. Gail,et al.  Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. , 1989, Journal of the National Cancer Institute.

[19]  Margaret Pepe,et al.  Measures to Summarize and Compare the Predictive Capacity of Markers , 2009, The international journal of biostatistics.

[20]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.