EVALUATION AND COMPUTATION OF DIAGNOSTIC TESTS : A SIMPLE ALTERNATIVE

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance in medical science. When a test is based on an observed variable that lies on a continuous scale, an assessment of the overall value of the test can be made through the use of a Receiver Operating Characteristic (ROC) curve. The ROC curve descibes the discrimination ability of a diagnosis test for the diseased subjects from the non-diseased subjects. The area under the ROC curve (AUC) represents the probability that a randomly chosen diseased subject will have higher probability of having disease than a randomly chosen non-diseased subject. For comparing two diagnostic systems, the difference between AUCs is often used. In this paper we have investigated various methods of the comparison of equality of two AUCs and proposed a McNemar test for the comparison of two diagnostic test procedures. The proposed test is based on an optimal cut-off point that discriminates the individuals in actually positive or actually negative cases for which we have a 2× 2 contingency table where we can apply the McNemar test. The operating characteristics of the proposed test are evaluated using extensive simulation over a wide range of parameters.

[1]  Q. Mcnemar.  Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[2]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[3]  J. Swets.  Sensitivities and specificities of diagnostic tests. , 1982, JAMA.

[4]  Pranab Kumar Sen,et al.  On Some Convergence Properties of UStatistics , 1960 .

[5]  D. Dorfman,et al.  Maximum likelihood estimation of parameters of signal detection theory—A direct solution , 1968, Psychometrika.

[6]  D. Bamber.  The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .

[7]  M. Zweig,et al.  Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. , 1993, Clinical chemistry.

[8]  E. S. Venkatraman,et al.  A distribution-free procedure for comparing receiver operating characteristic curves from a paired experiment , 1996 .

[9]  Harry K Genant,et al.  On the non‐inferiority of a diagnostic test based on paired observations , 2003, Statistics in medicine.

[10]  N. Perkins,et al.  Optimal Cut-point and Its Corresponding Youden Index to Discriminate Individuals Using Pooled Blood Samples , 2005, Epidemiology.

[11]  Ingram Olkin,et al.  A Family of Bivariate Distributions Generated by the Bivariate Bernoulli Distribution , 1985 .

[12]  Aurent,et al.  A Bivariate Binary Model for Testing Dependence in Outcomes , 2012 .

[13]  Kurt Hornik,et al.  On the generation of correlated artificial binary data , 1998 .

[14]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[15]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .

[16]  R. F. Wagner,et al.  Continuous versus categorical data for ROC analysis: some quantitative considerations. , 2001, Academic radiology.

[17]  David Gur,et al.  A permutation test sensitive to differences in areas for comparing ROC curves from a paired design , 2005, Statistics in medicine.

[18]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[19]  W. Hoeffding.  A Class of Statistics with Asymptotically Normal Distribution , 1948 .