Extending the skill test for disease diagnosis.

For diagnostic tests, we present an extension to the skill plot introduced by Briggs and Zaretski (Biometrics 2008; 64:250-261). The method is motivated by diagnostic measures for osteopetrosis in a study summarized by Hans et al. (The Lancet 1996; 348:511-514). Diagnostic test accuracy is typically defined using the area (or partial area) under the receiver operator characteristic (ROC) curve. If partial area is used, the resulting statistic can be highly subjective because the focus region of the ROC curve corresponds to a set of low false-positive rates that are chosen by the experimenter. This paper introduces a more objective diagnostic test for which the focus region depends on a skill score, which in turn depends on the loss functions associated with misdiagnosis. More specifically, the skill-based diagnostic test serves as a more objective version of the nonparametric test introduced by Dodd and Pepe (Biometrics 2003; 59:614-623).

[1]  David Ruppert,et al.  Assessing the Skill of Yes/No Predictions , 2005, Biometrics.

[2]  Rob J Hyndman,et al.  Nonparametric confidence intervals for receiver operating characteristic curves , 2004 .

[3]  Gerda Claeskens,et al.  Empirical likelihood confidence regions for comparison distributions and roc curves , 2003 .

[4]  M. Pepe An Interpretation for the ROC Curve and Inference Using GLM Procedures , 2000, Biometrics.

[5]  Hugh Coe,et al.  Quantitative sampling using an Aerodyne aerosol mass spectrometer 1. Techniques of data interpretation and error analysis , 2003 .

[6]  M. Pepe The Statistical Evaluation of Medical Tests for Classification and Prediction , 2003 .

[7]  William M. Briggs,et al.  Skill in real-time solar wind shock forecasts , 2003 .

[8]  Chris Lloyd,et al.  Kernel estimators of the ROC curve are better than empirical , 1999 .

[9]  M. Schervish A General Method for Comparing Probability Assessors , 1989 .

[10]  G. Roussas A course in mathematical statistics , 1997 .

[11]  Lori E. Dodd,et al.  Partial AUC Estimation and Regression , 2003, Biometrics.

[12]  Thomas Lengauer,et al.  ROCR: visualizing classifier performance in R , 2005, Bioinform..

[13]  Margaret S Pepe,et al.  Using the ROC curve for gauging treatment effect in clinical trials , 2006, Statistics in medicine.

[14]  Russell Zaretzki,et al.  The Skill Plot: A Graphical Technique for Evaluating Continuous Diagnostic Tests , 2007, Biometrics.

[15]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[16]  B. Turnbull,et al.  NONPARAMETRIC AND SEMIPARAMETRIC ESTIMATION OF THE RECEIVER OPERATING CHARACTERISTIC CURVE , 1996 .

[17]  S G Baker,et al.  Identifying Combinations of Cancer Markers for Further Study as Triggers of Early Intervention , 2000, Biometrics.

[18]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[19]  G. Breart,et al.  Ultrasonographic heel measurements to predict hip fracture in elderly women: the EPIDOS prospective study , 1996, The Lancet.