A test for crossing receiver operating characteristic (roc) curves

The receiver operating characteristic (ROC) curve gives a graphical representation of sensitivity and specificity of a prediction model when varying the decision treshold on a diagnostic criterion. A classical test for comparing the overall accuracies for two models -1 and 2- is based on the difference between ROC curves areas - related to its standard error. This test is designed for the situation where ROC curve 1 caps ROC curve 2. Often both curves cross :in this paper, a new test, based on the integrated difference between the curves, is proposed to deal with this situation. In a simulation experiment, the new test was less powerful than the old test for detecting an overall superiority, but much more powerfull against the crossing alternative.

[1]  B. Efron.  Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods , 1981 .

[2]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[3]  D Commenges,et al.  [Use of ROC curves]. , 1986, Revue d'epidemiologie et de sante publique.

[4]  C. Metz,et al.  Statistical significance tests for binormal ROC curves , 1980 .

[5]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[6]  J. Hanley,et al.  Statistical Approaches to the Analysis of Receiver Operating Characteristic (ROC) Curves , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[7]  P Ducimetière,et al.  Comparison of receiver operating curves derived from the same population: a bootstrapping approach. , 1985, Computers and biomedical research, an international journal.

[8]  Elisa T. Lee,et al.  Use of relative operating characteristic analysis in epidemiology. A method for dealing with subjective judgement. , 1981, American journal of epidemiology.

[9]  D. Dorfman,et al.  Maximum likelihood estimation of parameters of signal detection theory—A direct solution , 1968, Psychometrika.

[10]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[11]  D. Norman,et al.  A COMPARISON OF DATA OBTAINED WITH DIFFERENT FALSE-ALARM RATES. , 1964, Psychological review.

[12]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[13]  J. Swets,et al.  Assessment of diagnostic technologies. , 1979, Science.

[14]  D. Berman,et al.  Noninvasive prediction of the angiographic extent of coronary artery disease after myocardial infarction: comparison of clinical, bicycle exercise electrocardiographic, and ventriculographic parameters. , 1984, Circulation.

[15]  Irwin Pollack,et al.  Sampling variability of the area under the ROC-curve and of d'e. , 1969 .