A General Regression Methodology for ROC Curve Estimation

A method for applying generalized ordinal regression models to categorical rating data to estimate and analyze receiver operating characteristic (ROC) curves is presented. These models permit parsimonious adjustment of ROC curve parameters for relevant covariates through two regression equations that correspond to location and scale. Particular shapes of ROC curves are interpreted in relation to the kind of covariates included in the two regressions. The model is shown to be flexible because it is not restricted to the assumption of binormality that is commonly employed in smoothed ROC curve estimation, although the binormal model is one particular form of the more general model. The new method provides a mechanism for pinpointing the effect that interobserver variability has on the ROC curve. It also allows for the adjustment of ROC curves for temporal variation and case mix, and provides a way to assess the incremental diagnostic value of a test. The new methodology is recommended because it substantially improves the ability to assess diagnostic tests using ROC curves. Key words: ROC curves; ordinal regression; technology assessment; diagnostic tests. (Med Decis Making 8:204-215, 1988)

[1]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[2]  C B Begg,et al.  Biases in the assessment of diagnostic tests. , 1987, Statistics in medicine.

[3]  B. McNeil,et al.  Probabilistic Sensitivity Analysis Using Monte Carlo Simulation , 1985, Medical decision making : an international journal of the Society for Medical Decision Making.

[4]  C B Begg,et al.  Methodology for the Differential Diagnosis of a Complex Data Set , 1983, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  T. Birdsall.  The Theory of Signal Detectability: ROC Curves and Their Character , 1966 .

[6]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[7]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[8]  J. Hanley.  The Robustness of the "Binormal" Assumptions Used in Fitting ROC Curves , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[9]  N. E. Breslow.  Statistical Methods in Cancer Research , 1986 .

[10]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[11]  B J McNeil,et al.  Computed tomography, ultrasound, and scintigraphy of the liver in patients with colon or breast carcinoma: a prospective comparison. , 1983, Radiology.

[12]  P. M. E. Altham,et al.  A NON‐PARAMETRIC MEASURE OF SIGNAL DISCRIMINABILITY , 1973 .

[13]  J. C. Ogilvie,et al.  Maximum-likelihood estimation of receiver operating characteristic curve parameters , 1968 .

[14]  J. Swets,et al.  Assessment of diagnostic technologies. , 1979, Science.

[15]  J. Swets.  ROC analysis applied to the evaluation of medical imaging techniques. , 1979, Investigative radiology.

[16]  Anthony C. Atkinson,et al.  Regression Diagnostics, Transformations and Constructed Variables , 1982 .

[17]  J. Neyman,et al.  Research Papers in Statistics. Festschrift for J. Neyman F.N. David editor, assisted by E. Fix. London, New York, Sydney, J. Wiley & Sons, 1966, VIII p. 468 p., 105/–. , 1968, Recherches économiques de Louvain.

[18]  J. Swets.  Indices of discrimination or diagnostic accuracy: their ROCs and implied models. , 1986, Psychological bulletin.

[19]  J A Hanley,et al.  Response to ROC Steady , 1987, Medical decision making : an international journal of the Society for Medical Decision Making.

[20]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[21]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[22]  John A. Swets,et al.  Evaluation of diagnostic systems : methods from signal detection theory , 1982 .

[23]  P Doubilet,et al.  Interpretation of radiographs: effect of clinical history. , 1981, AJR. American journal of roentgenology.

[24]  R A Greenes,et al.  Assessment of diagnostic tests when disease verification is subject to selection bias. , 1983, Biometrics.

[25]  J A Swets,et al.  Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. , 1986, Psychological bulletin.