Comparing the Areas under More Than Two Independent ROC Curves

For a diagnostic test, the area under the associated receiver operating characteristic (ROC) curve is considered a measure of the efficacy of the test. Statistical methodology for the comparison of the areas under more than two independent ROC curves is developed. The jackknife is used to devise an F test using the pseudovalues as data. A Studentized range (SR) test is also considered using the original area estimates. A Monte Carlo study is performed to evaluate the significance level and power of the two test statistics. Both statistics conform well to the 0.10, 0.05, and 0.01 significance levels when the sampling design is balanced between cases with and without the disease. Power is also comparable. For unbalanced designs, the SR test on the original area estimates is very conservative while the F test on pseudovalues performs well. The F test is recommended as the method of choice for comparing the areas, although for balanced designs the SR test, with its com putational simplicity, may be preferred.

[1]  B Jennett,et al.  Assessment of outcome after severe brain damage. , 1975, Lancet.

[2]  R. G. Cornell,et al.  Improvements in burn care, 1965 to 1979. , 1980, JAMA.

[3]  T. Killip,et al.  Treatment of myocardial infarction in a coronary care unit. A two year experience with 250 patients. , 1967, The American journal of cardiology.

[4]  Application of Receiver-Operator Analysis to Diagnostic Tests of Iron Deficiency in Man , 1984, Pediatric Research.

[5]  J. Birch.  Introduction to the Theory of Statistics, 2nd Edition (Alexander M. Moon and Franklin A. Graybill) , 1964 .

[6]  L. Shaw,et al.  Diagnostic accuracy of four assays of prostatic acid phosphatase. Comparison using receiver operating characteristic curve analysis. , 1985, JAMA.

[7]  Williams Ln.  Doctor to disclose driver's identity. , 1974 .

[8]  B. Pruitt,et al.  Improvements in burn care. , 1980, JAMA.

[9]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[10]  Foss Mv.  MANAGEMENT OF VIRUS HEPATITIS. , 1964 .

[11]  Rupert G. Miller.  The jackknife-a review , 1974 .

[12]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[13]  B. Jennett,et al.  ASSESSMENT OF OUTCOME AFTER SEVERE BRAIN DAMAGE A Practical Scale , 1975, The Lancet.

[14]  J. Swets,et al.  Assessment of diagnostic technologies. , 1979, Science.

[15]  R. Galen,et al.  The Assessment of Laboratory Tests in the Diagnosis of Acute Appendicitis , 1984 .

[16]  F. Alcorn,et al.  The training of nonphysician personnel for use in a mammography program , 1969, Cancer.

[17]  C. Franklin,et al.  Clinical characteristics and resource utilization of ICU patients: Implications for organization of intensive care , 1987, Critical care medicine.

[18]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[19]  D. A. Berman,et al.  Treatment of myocardial infarction in a coronary care unit. , 1969, Minnesota medicine.

[20]  D. E. Lawrence,et al.  APACHE—acute physiology and chronic health evaluation: a physiologically based classification system , 1981, Critical care medicine.

[21]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[22]  C. Metz,et al.  Statistical significance tests for binormal ROC curves , 1980 .

[23]  William A. Knaus,et al.  Initial International Use of APACHE , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[24]  G. Teasdale.  Management of head injuries. , 1982, The Practitioner.

[25]  D. Bamber.  The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .

[26]  D. Berwick,et al.  Receiver Operating Characteristic Analysis of Diagnostic Skill , 1983, Medical care.

[27]  When is intensive care inappropriate? New "prognostic" measures provide answers. , 1986, Health management quarterly : HMQ.

[28]  Comparison of the effectiveness of four clinical chemical assays in classifying patients with chest pain. , 1982, Clinical chemistry.

[29]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[30]  R. Norris,et al.  A new coronary prognostic index. , 1970, Lancet.

[31]  Elisa T. Lee,et al.  Use of relative operating characteristic analysis in epidemiology. A method for dealing with subjective judgement. , 1981, American journal of epidemiology.