The "proper" binormal model: parametric receiver operating characteristic curve estimation with degenerate data.

RATIONALE AND OBJECTIVES The authors assessed the use of a "proper" binormal model and a new algorithm for maximum-likelihood estimation of receiver operating characteristic (ROC) curves from degenerate data. METHODS The proper binormal ROC model uses as its decision variable a monotonic transformation of the likelihood ratio that is associated with a pair of normal distributions, thereby ensuring fitted ROC curves with monotonic slope but maintaining a relationship with the conventional binormal model. A computer program entitled PROPROC was used to fit proper ROC curves to data obtained from computer-simulated and real observer studies. RESULTS ROC indexes such as total area were estimated with PROPROC and compared with the corresponding values obtained from the conventional procedures. CONCLUSION The proper binormal ROC model overcomes the problem of degeneracy in ROC curve fitting. PROPROC is highly robust and yields ROC estimates with less bias and greater precision than those obtained with the conventional binormal model.

[1]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[2]  K S Berbaum,et al.  Degeneracy and discrete receiver operating characteristic rating data. , 1995, Academic radiology.

[3]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[4]  David Gur,et al.  Statistical issues in ROC curve analysis , 1990, Medical Imaging.

[5]  H E Rockette,et al.  Nonparametric estimation of degenerate ROC data sets used for comparison of imaging systems. , 1990, Investigative radiology.

[6]  H E Rockette,et al.  On the generalization of the receiver operating characteristic analysis to the population of readers and cases with the jackknife method: an assessment. , 1995, Academic radiology.

[7]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[8]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[9]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[10]  C. Metz,et al.  A receiver operating characteristic partial area index for highly sensitive diagnostic tests. , 1996, Radiology.

[11]  M L Giger,et al.  Pulmonary nodules: computer-aided detection in digital chest images. , 1990, Radiographics : a review publication of the Radiological Society of North America, Inc.

[12]  Joseph H. Tashjian.  Proceedings of the Chest Imaging Conference 1987 , 1989 .

[13]  C E Metz,et al.  Some practical issues of experimental design and data analysis in radiological ROC studies. , 1989, Investigative radiology.

[14]  J A Swets,et al.  Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. , 1986, Psychological bulletin.

[15]  J. Hanley.  The Robustness of the "Binormal" Assumptions Used in Fitting ROC Curves , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[16]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[17]  John A. Swets,et al.  Evaluation of diagnostic systems : methods from signal detection theory , 1982 .

[18]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[19]  D. McClish.  Analyzing a Portion of the ROC Curve , 1989, Medical decision making : an international journal of the Society for Medical Decision Making.

[20]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[21]  R. Brent.  Table errata: Algorithms for minimization without derivatives (Prentice-Hall, Englewood Cliffs, N. J., 1973) , 1975 .

[22]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[23]  D. Bamber.  The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .

[24]  K. Berbaum,et al.  Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. , 1992, Investigative radiology.

[25]  K. Berbaum,et al.  Proper receiver operating characteristic analysis: the bigamma model. , 1997, Academic radiology.