Receiver Operator characteristic (ROC) Analysis without Truth

Receiver operator characteristic (ROC) analysis, the preferred method of evaluating diag nostic imaging tests, requires an independent assessment of the true state of disease, which can be difficult to obtain and is often of questionable accuracy. A new method of analysis is described which does not require independent truth data and which can be used when several accurate tests are being compared. This method uses correlative information to estimate the underlying model of multivariate normal distributions of disease-positive and disease-negative patients. The method is shown to give results equivalent to conventional ROC analysis in a comparison of computed tomography, radionuclide scintigraphy, and magnetic resonance imaging for liver metastasis. When independent truth is available, the method can be extended to incorporate truth data or to evaluate the consistency of the truth data with the imaging data. Key words: receiver operator characteristics; diagnostic evalu ation. (Med Decis Making 1990;10:24-29)

[1]  D. Dorfman,et al.  Maximum likelihood estimation of parameters of signal detection theory—A direct solution , 1968, Psychometrika.

[2]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[3]  E. Keeler,et al.  Primer on certain elements of medical decision making. , 1975, The New England journal of medicine.

[4]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[5]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[6]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[7]  B J McNeil,et al.  Computed tomography, ultrasound, and scintigraphy of the liver in patients with colon or breast carcinoma: a prospective comparison. , 1983, Radiology.

[8]  P. Guignard,et al.  Validity of the Gaussian assumption in the analysis of ROC data obtained from scintigraphic-like images. , 1983, Physics in medicine and biology.

[9]  C. Metz,et al.  A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data , 1984 .

[10]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[11]  Magnetic resonance imaging, computed tomography, and radionuclide scintigraphy in detection of liver metastases. , 1988, Canadian Association of Radiologists journal = Journal l'Association canadienne des radiologistes.