Evaluation of diagnostic accuracy in detecting ordered symptom statuses without a gold standard.

Our research is motivated by 2 methodological problems in assessing diagnostic accuracy of traditional Chinese medicine (TCM) doctors in detecting a particular symptom whose true status has an ordinal scale and is unknown-imperfect gold standard bias and ordinal scale symptom status. In this paper, we proposed a nonparametric maximum likelihood method for estimating and comparing the accuracy of different doctors in detecting a particular symptom without a gold standard when the true symptom status had an ordered multiple class. In addition, we extended the concept of the area under the receiver operating characteristic curve to a hyper-dimensional overall accuracy for diagnostic accuracy and alternative graphs for displaying a visual result. The simulation studies showed that the proposed method had good performance in terms of bias and mean squared error. Finally, we applied our method to our motivating example on assessing the diagnostic abilities of 5 TCM doctors in detecting symptoms related to Chills disease.

[1]  Yinsheng Qu,et al.  A Model for Evaluating Sensitivity and Specificity for Correlated Diagnostic Tests in Efficacy Studies with an Imperfect Reference Test , 1998 .

[2]  Nevin Lianwen Zhang,et al.  Hierarchical latent class models for cluster analysis , 2002, J. Mach. Learn. Res..

[3]  Todd A Alonzo,et al.  ROC Graphs for Assessing the Ability of a Diagnostic Marker to Detect Three Disease Classes with an Umbrella Ordering , 2007, Biometrics.

[4]  A. Hadgu,et al.  A biomedical application of latent class models with random effects , 2002 .

[5]  T. Kaptchuk.  The Web That Has No Weaver : Understanding Chinese Medicine , 1983 .

[6]  José Hernández-Orallo,et al.  Volume under the ROC Surface for Multi-class Problems , 2003, ECML.

[7]  Xiao-Hua Zhou,et al.  ROC Surfaces in the Presence of Verification Bias , 2007 .

[8]  Tao Chen,et al.  Hierarchical Latent Class Models and Statistical Foundation for Traditional Chinese Medicine , 2007, AIME.

[9]  D. Mossman.  Three-way ROCs , 1999, Medical decision making : an international journal of the Society for Medical Decision Making.

[10]  M. Bronskill,et al.  Receiver Operator characteristic (ROC) Analysis without Truth , 1990, Medical decision making : an international journal of the Society for Medical Decision Making.

[11]  Xiao-Hua Zhou,et al.  Nonparametric Estimation of ROC Curves in the Absence of a Gold Standard , 2005, Biometrics.

[12]  Tomas Kocka,et al.  Efficient learning of hierarchical latent class models , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[13]  S. Walter,et al.  Estimating the error rates of diagnostic tests. , 1980, Biometrics.

[14]  Zhaohui Wu,et al.  Knowledge discovery in traditional Chinese medicine: State of the art and perspectives , 2006, Artif. Intell. Medicine.

[15]  Xiao-Hua Zhou,et al.  NONPARAMETRIC ESTIMATION OF COMPONENT DISTRIBUTIONS IN A MULTIVARIATE MIXTURE , 2003 .

[16]  Mary Lou Thompson,et al.  Assessing the diagnostic accuracy of a sequence of tests. , 2003, Biostatistics.

[17]  S. Hui,et al.  Evaluation of diagnostic tests without gold standards , 1998, Statistical methods in medical research.

[18]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[19]  Tom Fawcett,et al.  Using rule sets to maximize ROC performance , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[20]  ChenTao,et al.  Latent tree models and diagnosis in traditional Chinese medicine , 2008 .

[21]  References , 1971 .

[22]  C. Yiannoutsos,et al.  Ordered multiple‐class ROC analysis with continuous measurements , 2004, Statistics in medicine.

[23]  Tao Chen,et al.  Latent tree models and diagnosis in traditional Chinese medicine , 2008, Artif. Intell. Medicine.