"Proper" Binormal ROC Curves: Theory and Maximum-Likelihood Estimation.

The conventional binormal model, which assumes that a pair of latent normal decision-variable distributions underlies ROC data, has been used successfully for many years to fit smooth ROC curves. However, if the conventional binormal model is used for small data sets or ordinal-category data with poorly allocated category boundaries, a "hook" in the fitted ROC may be evident near the upper-right or lower-left corner of the unit square. To overcome this curve-fitting artifact, we developed a "proper" binormal model and a new algorithm for maximum-likelihood (ML) estimation of the corresponding ROC curves. Extensive simulation studies have shown the algorithm to be highly reliable. ML estimates of the proper and conventional binormal ROC curves are virtually identical when the conventional binormal ROC shows no "hook," but the proper binormal curves have monotonic slope for all data sets, including those for which the conventional model produces degenerate fits. Copyright 1999 Academic Press.

[1]  Calyampudi R. Rao,et al.  Advanced Statistical Methods in Biometric Research. , 1953 .

[2]  J SWETS,et al.  Decision processes in perception. , 1961, Psychological review.

[3]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[4]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[5]  T. Birdsall.  The Theory of Signal Detectability: ROC Curves and Their Character , 1966 .

[6]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[7]  Calyampudi R. Rao,et al.  Advanced Statistical Methods in Biometric Research. , 1953 .

[8]  Elijah Polak,et al.  Computational methods in optimization , 1971 .

[9]  Byron J. T. Morgan,et al.  Some aspects of ROC curve-fitting: Normal and logistic models , 1972 .

[10]  A. Simpson,et al.  What is the best index of detectability? , 1973, Psychological Bulletin.

[11]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[12]  R. Brent.  Table errata: Algorithms for minimization without derivatives (Prentice-Hall, Englewood Cliffs, N. J., 1973) , 1975 .

[13]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[14]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[15]  John A. Swets,et al.  Evaluation of diagnostic systems : methods from signal detection theory , 1982 .

[16]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[17]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[18]  J A Swets,et al.  Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. , 1986, Psychological bulletin.

[19]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[20]  J. Hanley.  The Robustness of the "Binormal" Assumptions Used in Fitting ROC Curves , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[21]  D. McClish.  Analyzing a Portion of the ROC Curve , 1989, Medical decision making : an international journal of the Society for Medical Decision Making.

[22]  Joseph H. Tashjian.  Proceedings of the Chest Imaging Conference 1987 , 1989 .

[23]  C E Metz,et al.  Some practical issues of experimental design and data analysis in radiological ROC studies. , 1989, Investigative radiology.

[24]  William H. Press,et al.  Numerical recipes , 1990 .

[25]  H E Rockette,et al.  Nonparametric estimation of degenerate ROC data sets used for comparison of imaging systems. , 1990, Investigative radiology.

[26]  K S Berbaum,et al.  Degeneracy and discrete receiver operating characteristic rating data. , 1995, Academic radiology.

[27]  C. Metz,et al.  Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. , 1998, Statistics in medicine.

[28]  H. V. Trees.  Detection, Estimation, And Modulation Theory , 2001 .