Nonparametric estimation of ROC curves based on Bayesian models when the true disease state is unknown

We develop a Bayesian methodology for nonparametric estimation of ROC curves used for evaluation of the accuracy of a diagnostic procedure. We consider the situation where there is no perfect reference test, that is, no “gold standard”. The method is based on a multinomial model for the joint distribution of test-positive and test-negative observations. We use a Bayesian approach which assures the natural monotonicity property of the resulting ROC curve estimate. MCMC methods are used to compute the posterior estimates of the sensitivities and specificities that provide the basis for inference concerning the accuracy of the diagnostic procedure. Because there is no gold standard, identifiability requires that the data come from at least two populations with different prevalences. No assumption is needed concerning the shape of the distributions of test values of the diseased and non diseased in these populations. We discuss an application to an analysis of ELISA scores in the diagnostic testing of paratuberculosis (Johne’s Disease) for several herds of dairy cows and compare the results to those obtained from some previously proposed methods.

[1]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[2]  S. Hui,et al.  Evaluation of diagnostic tests without gold standards , 1998, Statistical methods in medical research.

[3]  J. Stabel.  Transitions in immune responses to Mycobacterium paratuberculosis. , 2000, Veterinary microbiology.

[4]  Søren Højsgaard,et al.  Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. , 2005, Preventive veterinary medicine.

[5]  Wesley O. Johnson,et al.  Correlation‐adjusted estimation of sensitivity and specificity of two diagnostic tests , 2003 .

[6]  W O Johnson,et al.  Screening without a "gold standard": the Hui-Walter paradigm revisited. , 2001, American journal of epidemiology.

[7]  S D Walter,et al.  Estimation of test error rates, disease prevalence and relative risk from misclassified data: a review. , 1988, Journal of clinical epidemiology.

[8]  M. Bronskill,et al.  Receiver Operator characteristic (ROC) Analysis without Truth , 1990, Medical decision making : an international journal of the Society for Medical Decision Making.

[9]  B. Craig,et al.  Estimating disease prevalence in the absence of a gold standard , 2002, Statistics in medicine.

[10]  M. Pepe.  The Statistical Evaluation of Medical Tests for Classification and Prediction , 2003 .

[11]  Wesley O. Johnson,et al.  Hierarchical models for estimating herd prevalence and test accuracy in the absence of a gold standard , 2003 .

[12]  R. F. Wagner,et al.  The problem of ROC analysis without truth: the EM algorithm and the information matrix , 2000, Medical Imaging.

[13]  I Yang,et al.  Latent variable modeling of diagnostic accuracy. , 1997, Biometrics.

[14]  S. Walter,et al.  Estimating the error rates of diagnostic tests. , 1980, Biometrics.

[15]  Scott Zeger,et al.  Methods for evaluating the performance of diagnostic tests in the absence of a gold standard: a latent class model approach , 2002, Statistics in medicine.

[16]  M. Tan,et al.  Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. , 1996, Biometrics.

[17]  Paul Gustafson,et al.  The utility of prior information and stratification for parameter estimation with two screening tests but no gold standard , 2005, Statistics in medicine.

[18]  P. Albert,et al.  A Cautionary Note on the Robustness of Latent Class Models for Estimating Diagnostic Error without a Gold Standard , 2004, Biometrics.

[19]  Walter R. Gilks,et al.  BUGS - Bayesian inference Using Gibbs Sampling Version 0.50 , 1995 .

[20]  I A Gardner,et al.  Estimation of diagnostic-test sensitivity and specificity through Bayesian modeling. , 2005, Preventive veterinary medicine.

[21]  C B Begg,et al.  Consensus Diagnoses and "Gold Standards" , 1990, Medical decision making : an international journal of the Society for Medical Decision Making.

[22]  Claes Enùea,et al.  Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown , 2000 .

[23]  Xiao-Hua Zhou,et al.  NONPARAMETRIC ESTIMATION OF COMPONENT DISTRIBUTIONS IN A MULTIVARIATE MIXTURE , 2003 .

[24]  Xiao-Hua Zhou,et al.  Nonparametric Estimation of ROC Curves in the Absence of a Gold Standard , 2005, Biometrics.

[25]  M. Greiner,et al.  Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. , 2000, Preventive veterinary medicine.

[26]  S. Nielsen,et al.  Maximum-likelihood estimation of sensitivity and specificity of ELISAs and faecal culture for diagnosis of paratuberculosis. , 2002, Preventive veterinary medicine.

[27]  L. Joseph,et al.  Bayesian Approaches to Modeling the Conditional Dependence Between Multiple Diagnostic Tests , 2001, Biometrics.

[28]  S Andersen,et al.  Re: "Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard". , 1997, American journal of epidemiology.

[29]  Wesley O. Johnson,et al.  Bayesian inferences for receiver operating characteristic curves in the absence of a gold standard , 2006 .

[30]  L. Joseph,et al.  Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. , 1995, American journal of epidemiology.

[31]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.