Determination and Interpretation of the Optimal Operating Point for ROC Curves Derived Through Generalized Linear Models

Sensitivity and specificity summarize the performance of a diagnostic test with a positive/negative outcome determined by a gold standard. When the test is quantitative, receiver operating characteristic (ROC) curves are used to display the performance of all possible cutpoints of the quantitative diagnostic marker. The ROC curve offers a graphical interpretation of the trade-off between sensitivity and specificity of the range of possible cutpoints. Various methods are used to estimate the ROC curve including empirical, parametric, semiparametric, and regression methods. Similarly, various software packages provide ROC curve estimation. Regression methods based on using generalized linear models are discussed as well as their implementation with the SAS procedures PROC LOGISTIC and PROC GENMOD. Recent attention has been given to determining the optimal decision rule, also called the optimal operating point (OOP). Similar to the ROC curve, the OOP provides a graphical interpretation for decision making. T...

[1]  J. Hanley,et al.  The comparison of injury severity instrument performance using likelihood ratio and ROC curve analyses. , 1995, The Journal of trauma.

[2]  M. Zweig,et al.  Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. , 1993, Clinical chemistry.

[3]  Andrew D. A. Maidment,et al.  Comparison of receiver operating characteristic curves on the basis of optimal operating points. , 1996, Academic radiology.

[4]  M Schulzer,et al.  Diagnostic tests: A statistical review , 1994, Muscle & nerve.

[5]  H. Schäfer,et al.  Constructing a cut-off point for a quantitative diagnostic test. , 1989, Statistics in medicine.

[6]  Margaret S. Pepe,et al.  A regression modelling framework for receiver operating characteristic curves in medical diagnostic testing , 1997 .

[7]  W. England,et al.  An Exponential Model Used for optimal Threshold selection on ROC Curues , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  A. Toledano,et al.  Ordinal regression methodology for ROC curves derived from correlated data. , 1996, Statistics in medicine.

[9]  A. Mushlin,et al.  Focusing Technology Assessment Using Medical Decision Theory , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[10]  W. Blattner,et al.  A method for predicting individual HIV infection status in the absence of clinical information. , 1988, AIDS research and human retroviruses.

[11]  D. Pregibon.  Goodness of Link Tests for Generalized Linear Models , 1980 .

[12]  L. Joseph,et al.  Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. , 1995, American journal of epidemiology.

[13]  C. Metz,et al.  Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. , 1998, Statistics in medicine.

[14]  T. Habuchi,et al.  Ratio of prostate specific antigen minor molecular forms-to-total prostate specific antigen is constant regardless of the pathological condition of the prostate. , 2003, The Journal of urology.

[15]  M S Pepe,et al.  Three approaches to regression analysis of receiver operating characteristic curves for continuous test results. , 1998, Biometrics.

[16]  S. Weisberg,et al.  ARES plots for generalized linear models , 1994 .

[17]  Scott L. Zeger,et al.  Latent Variable Regression for Multiple Discrete Outcomes , 1997 .

[18]  D L Riddle,et al.  Interpreting validity indexes for diagnostic tests: an illustration using the Berg balance test. , 1999, Physical therapy.

[19]  Yinsheng Qu,et al.  A Model for Evaluating Sensitivity and Specificity for Correlated Diagnostic Tests in Efficacy Studies with an Imperfect Reference Test , 1998 .

[20]  C. Y. Peng,et al.  Logistic Regression Analysis and Reporting: A Primer , 2002 .

[21]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[22]  Joseph L. Gastwirth,et al.  Bayesian Inference for Medical Screening Tests: Approximations Useful for the Analysis of Acquired Immune Deficiency Syndrome , 1991 .

[23]  D. Rindskopf,et al.  The value of latent class analysis in medical diagnosis. , 1986, Statistics in medicine.

[24]  S. Sahadevan,et al.  Diagnostic performance of two mental status tests in the older Chinese: influence of education and age on cut‐off values , 2000, International journal of geriatric psychiatry.

[25]  R. Dennis Cook,et al.  Regression Diagnostics With Dynamic Graphics , 1989 .

[26]  Roy E. Welsch,et al.  Regression Diagnostics with Dynamic Graphics: [With Discussions and Response] , 1989 .

[27]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[28]  M. Tan,et al.  Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. , 1996, Biometrics.

[29]  P N Valenstein,et al.  Evaluating diagnostic tests with imperfect standards. , 1990, American journal of clinical pathology.

[30]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[31]  Lloyd D. Fisher,et al.  2. Biostatistics: A Methodology for the Health Sciences , 1994 .

[32]  M. Pepe.  An Interpretation for the ROC Curve and Inference Using GLM Procedures , 2000, Biometrics.

[33]  P. Ubel,et al.  Who will enroll? Predicting participation in a phase II AIDS vaccine trial. , 2001 .

[34]  H. Ishwaran,et al.  A general class of hierarchical ordinal regression models with applications to correlated roc analysis , 2000 .

[35]  D. Mossman.  Three-way ROCs , 1999, Medical decision making : an international journal of the Society for Medical Decision Making.

[36]  J R Fielding,et al.  Bayesian regression methodology for estimating a receiver operating characteristic curve with two radiologic applications: prostate biopsy and spiral CT of ureteral stones. , 2001, Academic radiology.

[37]  S. Soyupak,et al.  Osteopenia and osteoporosis in postmenopausal women. Assessment by radiogrammetric measurement. , 2002, Acta radiologica.

[38]  M. Gladis,et al.  Early improvement during manual-guided cognitive and dynamic psychotherapies predicts 16-week remission status. , 2001, The Journal of psychotherapy practice and research.

[39]  D. Restrepo,et al.  Characteristics of odorant elicited calcium changes in cultured human olfactory neurons , 2000, Journal of neuroscience research.

[40]  Els Goetghebeur,et al.  Diagnostic test analyses in search of their gold standard: latent class analyses with random effects , 2000, Statistical methods in medical research.

[41]  F Sainfort,et al.  Evaluation of medical technologies: a generalized ROC analysis. , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[42]  E. Somoza,et al.  Comparing and Optimizing Diagnostic Tests , 1992, Medical decision making : an international journal of the Society for Medical Decision Making.

[43]  B. Turnbull,et al.  NONPARAMETRIC AND SEMIPARAMETRIC ESTIMATION OF THE RECEIVER OPERATING CHARACTERISTIC CURVE , 1996 .

[44]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[45]  M. Binder,et al.  Comparing Three-class Diagnostic Tests by Three-way ROC Analysis , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[46]  D. McClish,et al.  Comparing the Areas under More Than Two Independent ROC Curves , 1987, Medical decision making : an international journal of the Society for Medical Decision Making.

[47]  A. Beck,et al.  Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation , 1988 .

[48]  Walter W. Piegorsch,et al.  Complementary Log Regression for Generalized Linear Models , 1992 .

[49]  S D Walter,et al.  Effects of dependent errors in the assessment of diagnostic test performance. , 1997, Statistics in medicine.

[50]  John S. Uebersax,et al.  Probit Latent Class Analysis with Dichotomous or Ordered Category Measures: Conditional Independence/Dependence Models , 1999 .

[51]  D V Canter,et al.  Linking commercial burglaries by modus operandi: tests using regression and ROC analysis. , 2002, Science & justice : journal of the Forensic Science Society.

[52]  D. Bamber.  The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .