Comparison of receiver operating curves derived from the same population: a bootstrapping approach.

The receiver operating curve (ROC) gives a representation of sensitivity and specificity of a prediction model when varying the cutpoint of a decision rule on a whole spectrum. Evaluation of two models established (or tested) in the same population of patients warrants a valid statistical comparison of their ROC curves. Hanley et al. recently provided a method for overall comparison of ROC curves (J. A. Hanley and B. J. McNeil, Radiology 148, 839-843, 1983). Often ROC curves cross, or differ in only a part of their courses. Bootstrapping of ROC curves is proposed as a graphical check for the statistical significance of differences confined to a part of the curve. An example comparing two models of prediction of coronary artery disease progression is given to illustrate this new approach.

[1]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[2]  D. Norman,et al.  A COMPARISON OF DATA OBTAINED WITH DIFFERENT FALSE-ALARM RATES. , 1964, Psychological review.

[3]  Yves Taeymans,et al.  Clinical and Angiographic Factors Associated With Coronary Artery Disease Progression of Coronary Artery Disease , 1984 .

[4]  S. Pauker,et al.  The relative value of carotid noninvasive testing as determined by receiver operator characteristic curves. , 1980, Surgery.

[5]  A. Bruschke,et al.  The Anatomic Evolution of Coronary Artery Disease Demonstrated by Coronary Arteriography in 256 Nonoperated Patients , 1981, Circulation.

[6]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[7]  Gail Gong.  Cross-Validation, the Jackknife, and the Bootstrap: Excess Error Estimation in Forward Logistic Regression , 1986 .

[8]  B. Efron.  Nonparametric standard errors and confidence intervals , 1981 .

[9]  B J McNeil,et al.  Statistical approaches to clinical predictions. , 1981, The New England journal of medicine.

[10]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[11]  C E Metz,et al.  Critical evaluation of stress testing in the diagnosis of peripheral vascular disease. , 1982, Surgery.

[12]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[13]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[14]  J. Kramer,et al.  Progression and regression of coronary atherosclerosis: relation to risk factors. , 1983, American heart journal.

[15]  Irwin Pollack,et al.  Sampling variability of the area under the ROC-curve and of d'e. , 1969 .