Assessing new biomarkers and predictive models for use in clinical practice: a clinician's guide.

New biomarkers and predictive models that aim to improve the identification of people at risk of cardiovascular disease are constantly proposed. Clinicians need to be aware of the various methods used to assess these biomarkers and models and how these should be interpreted. New biomarkers and models are assessed in terms of their contribution to global fit, discrimination, calibration, and reclassification. These measures, when used in isolation, do not address the clinically important questions of whether the new model predicts risk more accurately than existing models and whether the risks predicted for individuals are sufficiently different to warrant a change in treatment decisions. We recommend that these measures be supplemented with graphical displays such as a calibration plot for the Hosmer-Lemeshow test and a scatterplot of the risks predicted by the models being compared. We encourage researchers to report such analyses from studies on the clinical utility of new biomarkers because this information is pertinent for the clinician who must decide whether to test for a new biomarker in their clinical practice.

[1]  N. Unwin,et al.  Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) Detection, Evaluation, and Treatment of High Blood Cholesterol Education Program (NCEP) Expert Panel on Executive Summary of the Third Report of the National , 2009 .

[2]  W. Winkelmayer,et al.  Comparison of cardiovascular outcomes in elderly patients with diabetes who initiated rosiglitazone vs pioglitazone therapy. , 2008, Archives of internal medicine.

[3]  J. Sundström,et al.  Use of multiple biomarkers to improve the prediction of death from cardiovascular causes , 2008 .

[4]  R. Klein,et al.  Risk prediction of coronary heart disease based on retinal vascular caliber (from the Atherosclerosis Risk In Communities [ARIC] Study). , 2008, The American journal of cardiology.

[5]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[6]  Helena Chmura Kraemer,et al.  Comments on ‘Evaluating the added predictive ability of a new marker’ by M. Pencina, R. D'Agostino, R. D'Agostino Jr, R. Vasan, Statistics in Medicine (DOI: 10.1002/sim.2929) , 2008, Statistics in medicine.

[7]  Sander Greenland,et al.  The need for reorientation toward cost‐effective prediction: Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929) , 2008, Statistics in medicine.

[8]  J. Ware,et al.  Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929) , 2008, Statistics in medicine.

[9]  Yingye Zheng,et al.  Integrating the predictiveness of a marker with its performance as a classifier. , 2007, American journal of epidemiology.

[10]  J. Hippisley-Cox,et al.  Performance of the QRISK cardiovascular risk prediction algorithm in an independent UK sample of patients from general practice: a validation study , 2007, Heart.

[11]  J. Zimmerman,et al.  Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited* , 2007, Critical care medicine.

[12]  M. Pencina,et al.  Clinical utility of different lipid measures for prediction of coronary heart disease in men and women. , 2007, JAMA.

[13]  R. Kronmal,et al.  Implications of C-reactive protein or coronary artery calcium score as an adjunct to global risk assessment for primary prevention of CHD. , 2007, Atherosclerosis.

[14]  J. Hippisley-Cox,et al.  Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study , 2007, BMJ : British Medical Journal.

[15]  Rowena J Dolor,et al.  Evidence-based guidelines for cardiovascular disease prevention in women: 2007 update. , 2007, Journal of the American College of Cardiology.

[16]  N. Cook.  Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction , 2007, Circulation.

[17]  N. Cook,et al.  Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. , 2007, JAMA.

[18]  M. Woodward,et al.  Adding social deprivation and family history to cardiovascular risk assessment: the ASSIGN score from the Scottish Heart Health Extended Cohort (SHHEC) , 2005, Heart.

[19]  R. Klein,et al.  Quantitative retinal venular caliber and risk of cardiovascular disease in older persons: the cardiovascular health study. , 2006, Archives of internal medicine.

[20]  Nancy R Cook,et al.  The Effect of Including C-Reactive Protein in Cardiovascular Risk Prediction Models for Women , 2006, Annals of Internal Medicine.

[21]  Diabetes Uk,et al.  JBS 2: Joint British Societies9 guidelines on prevention of cardiovascular disease in clinical practice , 2005 .

[22]  J. Gibbs,et al.  JBS 2: Joint British Societies' guidelines on prevention of cardiovascular disease in clinical practice , 2005, Heart.

[23]  Michael W Kattan,et al.  Evaluating a New Marker’s Predictive Contribution , 2004, Clinical Cancer Research.

[24]  Calyampudi Radhakrishna Rao,et al.  Advances in Survival Analysis , 2003, Handbook of Statistics.

[25]  A. Folsom,et al.  Coronary heart disease risk prediction in the Atherosclerosis Risk in Communities (ARIC) study. , 2003, Journal of clinical epidemiology.

[26]  Sunil J Rao,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2003 .

[27]  Neil M Bressler,et al.  Retinal arteriolar narrowing and risk of coronary heart disease. , 2003, Archives of ophthalmology.

[28]  J. Mckenney,et al.  National Cholesterol Education Program Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) , 2002 .

[29]  S M Grundy,et al.  Improving coronary heart disease risk assessment in asymptomatic people: role of traditional risk factors and noninvasive cardiovascular tests. , 2001, Circulation.

[30]  J. Mckenney,et al.  Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III). , 2001, JAMA.

[31]  C. Rembold.  Number needed to screen: development of a statistic for disease screening , 1998, BMJ.

[32]  D. Levy,et al.  Prediction of coronary heart disease using risk factor categories. , 1998, Circulation.

[33]  D. Hosmer,et al.  A Simplified Method of Calculating an Overall Goodness-of-Fit Test for the Cox Proportional Hazards Model , 1998, Lifetime data analysis.

[34]  D. Hosmer,et al.  A comparison of goodness-of-fit tests for the logistic regression model. , 1997, Statistics in medicine.

[35]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[36]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[37]  Stanley Lemeshow,et al.  Goodness-of-Fit Testing for the Logistic Regression Model when the Estimated Probabilities are Small , 1988 .