Evaluation of Medical Images

From a practical point of view, image quality in medicine must be defined in terms of the decisons that physicians can make by reading the images. Image-based decisions concerning the actual state of an object or patient are evaluated most meaningfully by ROC analysis, which has been used to quantify detection performance in sensory psychophysics since the early 1960s and more recently has been applied to evaluate diagnostic systems. After surveying briefly some considerations that motivate the use of ROC analysis in medical image evauation, this manuscript addresses various practical issues of experimental design and data analysis, including the selection of cases and observers, the need for standards of truth, reading-order effects, data collection, curve fitting, statistical testing, and possible generalizations of conventional ROC methodology.

[1]  Byron J. T. Morgan,et al.  Some aspects of ROC curve-fitting: Normal and logistic models , 1972 .

[2]  J. Hanley,et al.  Statistical Approaches to the Analysis of Receiver Operating Characteristic (ROC) Curves , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[3]  C E Metz,et al.  Some practical issues of experimental design and data analysis in radiological ROC studies. , 1989, Investigative radiology.

[4]  M. Weinstein,et al.  Clinical Decision Analysis , 1980 .

[5]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[6]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[7]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[8]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[9]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[10]  Arthur G. Haus,et al.  The Physics of Medical Imaging: Recording System Measurements and Techniques , 1979 .

[11]  L B Lusted,et al.  General problems in medical decision making with comments on ROC analysis. , 1978, Seminars in nuclear medicine.

[12]  J. Swets.  Indices of discrimination or diagnostic accuracy: their ROCs and implied models. , 1986, Psychological bulletin.

[13]  C. Metz,et al.  Visual detection and localization of radiographic images. , 1975, Radiology.

[14]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[15]  D P Chakraborty,et al.  Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data. , 1989, Medical physics.

[16]  K. Ayyangar.  Medical Radionuclide Imaging , 1978 .

[17]  Charles E. Metz,et al.  Progress in evaluation of human observer visual detection performance using the ROC curve approach , 1976 .

[18]  C. Metz,et al.  Statistical significance tests for binormal ROC curves , 1980 .

[19]  N A Macmillan,et al.  Detection theory analysis of group data: estimating sensitivity from average hit and false-alarm rates. , 1985, Psychological bulletin.

[20]  E. Keeler,et al.  Primer on certain elements of medical decision making. , 1975, The New England journal of medicine.

[21]  B. McNeil,et al.  Assessment of radiologic tests: control of bias and other design considerations. , 1988, Radiology.

[22]  Kunio Doi,et al.  An Empirical Investigation Of Variability In Contrast-Detail Diagram Measurements , 1983, Other Conferences.

[23]  R A Greenes,et al.  Construction of Receiver Operating Characteristic Curves when Disease Verification Is Subject to Selection Bias , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[24]  Applications of ROC analysis in diagnostic image evaluation , 1979 .

[25]  C. Metz,et al.  A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data , 1984 .

[26]  John F. Hamilton,et al.  A Free Response Approach To The Measurement And Characterization Of Radiographic Observer Performance , 1977, Other Conferences.

[27]  A. Feinstein,et al.  Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. , 1978, The New England journal of medicine.

[28]  J. Swets.  Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. , 1986, Psychological bulletin.

[29]  J. Swets,et al.  Enhanced interpretation of diagnostic images. , 1988, Investigative radiology.

[30]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[31]  John A. Swets,et al.  Evaluation of diagnostic systems : methods from signal detection theory , 1982 .

[32]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.