Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups

This article studies the problem of measuring and estimating the diagnostic accuracy when there are three ordinal diagnostic groups. We use a receiver operating characteristic (ROC) surface to describe the probabilities of correct classifications into three diagnostic groups based on various sets of diagnostic thresholds of a test and propose to use the entire and the partial volume under the surface to measure the diagnostic accuracy. Mathematical properties and probabilistic interpretations of the proposed measure of diagnostic accuracy are discussed. Under the assumption of normal distributions of the diagnostic test from three diagnostic groups, we present the maximum likelihood estimate to the volume under the ROC surface and give the asymptotic variance to the estimate. We further propose several asymptotic confidence interval estimates to the volume under the ROC surface. The performance of these confidence interval estimates is evaluated in terms of attaining the nominal coverage probability based on a simulation study. In addition, we develop a method of sample size determination to achieve an adequate accuracy of the confidence interval estimate. Finally, we demonstrate the proposed methodology by applying it to the clinical diagnosis of early stage Alzheimer's disease based on the neuropsychological database of the Washington University Alzheimer's Disease Research Center. Copyright © 2005 John Wiley & Sons, Ltd.