Estimation of the Youden Index and its Associated Cutoff Point

The Youden Index is a frequently used summary measure of the ROC (Receiver Operating Characteristic) curve. It both, measures the effectiveness of a diagnostic marker and enables the selection of an optimal threshold value (cutoff point) for the marker. In this paper we compare several estimation procedures for the Youden Index and its associated cutoff point. These are based on (1) normal assumptions; (2) transformations to normality; (3) the empirical distribution function; (4) kernel smoothing. These are compared in terms of bias and root mean square error in a large variety of scenarios by means of an extensive simulation study. We find that the empirical method which is the most commonly used has the overall worst performance. In the estimation of the Youden Index the kernel is generally the best unless the data can be well transformed to achieve normality whereas in estimation of the optimal threshold value results are more variable. (© 2005 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

[1]  M. Greiner,et al.  Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. , 2000, Preventive veterinary medicine.

[2]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[3]  M. W. Thompson,et al.  Duchenne muscular dystrophy carrier detection using logistic discrimination: serum creatine kinase, hemopexin, pyruvate kinase, and lactate dehydrogenase in combination. , 1982, American journal of medical genetics.

[4]  I Hinberg,et al.  Receiver operator characteristic (ROC) curves and non-normal data: an empirical study. , 1990, Statistics in medicine.

[5]  K. Zou,et al.  Two transformation models for estimating an ROC curve derived from continuous data , 2000 .

[6]  Peter Dalgaard,et al.  Introductory statistics with R , 2002, Statistics and computing.

[7]  G S Challand.  Measurement in Laboratory Medicine. A Primer on Control and Interpretation , 1996 .

[8]  J J Bartko,et al.  Diagnosing diagnoses. Receiver operating characteristic methods and psychiatry. , 1989, Archives of general psychiatry.

[9]  Douglas Mossman,et al.  The INFO-ROC Technique: A Method for Comparing and Optimizing Inspection Systems , 1990 .

[10]  J R Fielding,et al.  Original smooth receiver operating characteristic curve estimation from continuous data: statistical methods for analyzing the predictive value of spiral CT of ureteral stones. , 1998, Academic radiology.

[11]  W. J. Hall,et al.  Semiparametric and parametric transformation models for comparing diagnostic markers with paired design , 2002 .

[12]  R. F.,et al.  Mathematical Statistics , 1944, Nature.

[13]  Rob J Hyndman,et al.  Improved methods for bandwidth selection when estimating ROC curves , 2003 .

[14]  C E Metz,et al.  Some practical issues of experimental design and data analysis in radiological ROC studies. , 1989, Investigative radiology.

[15]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[16]  P. Robinson.  The interpretation of diagnostic tests. , 1987, Nuclear medicine communications.

[17]  David Faraggi,et al.  Adjusting receiver operating characteristic curves and related indices for covariates , 2003 .

[18]  T. Xie,et al.  Evaluation of cutoff levels for screening of gastric cancer using serum pepsinogens and distributions of levels of serum pepsinogen I, II and of PG I/PG II ratios in a gastric cancer case-control study. , 1997, Journal of epidemiology.

[19]  S. Grmec,et al.  Comparison of APACHE II, MEES and Glasgow Coma Scale in patients with nontraumatic coma for prediction of mortality , 2000, Critical care.

[20]  B. Reiser,et al.  Estimation of the area under the ROC curve , 2002, Statistics in medicine.