Dual composite reference standards (dCRS) in molecular diagnostic research: A new approach to reduce bias in the presence of Imperfect reference

ABSTRACT A main challenge in molecular diagnostic research is to accurately evaluate the performance of a new nucleic acid amplification test when the reference standard is imperfect. Several approaches, such as discrepant analysis, composite reference standard (CRS) method, or latent class analysis (LCA), are commonly applied for this purpose by combining multiple imperfect (reference) test results. In discrepant analysis or LCA, test results from the new assay are often involved in the construction of a new pseudo-reference standard, which results in the potential risk of overestimating the parameters of interest. On the contrary, the CRS methods only combine the results of reference tests, which is more preferable in practice. In this article, we study the properties of two extreme CRS methods, i.e., combining multiple reference test results by the “any positive” rule or by the “all-positive” rule, and propose a new approach “dual composite reference standards (dCRS)” based on these two extreme methods to reduce the biases of the estimates. Simulations are performed for various scenarios and the proposed approach is applied to two real datasets. The results demonstrate that our approach outperforms other commonly used approaches and therefore is recommended for future applications.

[1]  Yinsheng Qu,et al.  A Model for Evaluating Sensitivity and Specificity for Correlated Diagnostic Tests in Efficacy Studies with an Imperfect Reference Test , 1998 .

[2]  S D Walter,et al.  Estimation of test error rates, disease prevalence and relative risk from misclassified data: a review. , 1988, Journal of clinical epidemiology.

[3]  Margaret Sullivan Pepe,et al.  Insights into latent class analysis of diagnostic test performance. , 2007, Biostatistics.

[4]  D. Rindskopf,et al.  The value of latent class analysis in medical diagnosis. , 1986, Statistics in medicine.

[5]  G. Alvarado-Urbina,et al.  Probe amplifier system based on chimeric cycling oligonucleotides. , 1990, BioTechniques.

[6]  R. Humphries,et al.  Performance of Clostridium difficile Toxin Enzyme Immunoassay and Nucleic Acid Amplification Tests Stratified by Patient Disease Severity , 2012, Journal of Clinical Microbiology.

[7]  J. Sellors,et al.  Bias in the evaluation of DNA-amplification tests for detecting Chlamydia trachomatis. , 1998, Statistics in medicine.

[8]  A Hadgu,et al.  Discrepant analysis: a biased and an unscientific method for estimating test sensitivity and specificity. , 1999, Journal of clinical epidemiology.

[9]  R. Yolken Nucleic acid amplification assays for microbial diagnosis: challenges and opportunities. , 2002, The Journal of pediatrics.

[10]  P. Bossuyt,et al.  Evaluation of diagnostic tests when there is no gold standard. A review of methods. , 2007, Health technology assessment.

[11]  D M Hawkins,et al.  Some issues in resolution of diagnostic tests using an imperfect gold standard , 2001, Statistics in medicine.

[12]  T. Gingeras,et al.  Transcription-based amplification system and detection of amplified human immunodeficiency virus type 1 with a bead-based sandwich hybridization format. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[13]  S. Hui,et al.  Evaluation of diagnostic tests without gold standards , 1998, Statistical methods in medical research.

[14]  W O Johnson,et al.  Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. , 2000, Preventive veterinary medicine.

[15]  M S Pepe,et al.  Using a combination of reference tests to assess the accuracy of a new diagnostic test. , 1999, Statistics in medicine.

[16]  Kurt Hornik,et al.  On the generation of correlated artificial binary data , 1998 .

[17]  A. Hadgu The discrepancy in discrepant analysis , 1996, The Lancet.

[18]  D. Y. Wu,et al.  The ligation amplification reaction (LAR)--amplification of specific DNA sequences using sequential rounds of template-dependent ligation. , 1989, Genomics.

[19]  Robert E. Johnson,et al.  Nucleic Acid Amplification Tests for Diagnosis of Neisseria gonorrhoeae Oropharyngeal Infections , 2009, Journal of Clinical Microbiology.

[20]  Schachter,et al.  Ligase chain reaction to detect Chlamydia trachomatis infection of the cervix , 1994, Journal of clinical microbiology.

[21]  K. Mullis,et al.  Primer-directed enzymatic amplification of DNA with a thermostable DNA polymerase. , 1988, Science.

[22]  L. A. Goodman Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[23]  Mellisa L. Theodore,et al.  Comparison of Three Nucleic Acid Amplification Tests for Detection of Chlamydia trachomatis in Urine Specimens , 2004, Journal of Clinical Microbiology.

[24]  William W. Thompson,et al.  Utility of Composite Reference Standards and Latent Class Analysis in Evaluating the Clinical Accuracy of Diagnostic Tests for Pertussis , 2007, Clinical and Vaccine Immunology.

[25]  A. Fauci,et al.  Infectious diseases: considerations for the 21st century. , 2001, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[26]  G. V. van Doornum,et al.  Detection of Chlamydia trachomatis and Neisseria gonorrhoeae by ligase chain reaction-based assays with clinical specimens from various sites: implications for diagnostic testing and screening , 1996, Journal of clinical microbiology.

[27]  P. Albert,et al.  A Cautionary Note on the Robustness of Latent Class Models for Estimating Diagnostic Error without a Gold Standard , 2004, Biometrics.

[28]  L. Joseph,et al.  Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. , 1995, American journal of epidemiology.

[29]  E. Harris,et al.  Development of an Internally Controlled Real-Time Reverse Transcriptase PCR Assay for Pan-Dengue Virus Detection and Comparison of Four Molecular Dengue Virus Detection Assays , 2013, Journal of Clinical Microbiology.

[30]  D. Böhning,et al.  Re: Insights into latent class analysis of diagnostic test performance. , 2008, Biostatistics.

[31]  Loes C M Bertens,et al.  Value of composite reference standards in diagnostic research , 2013, BMJ.

[32]  M. Tan,et al.  Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. , 1996, Biometrics.

[33]  Johannes B Reitsma,et al.  Bias due to composite reference standards in diagnostic accuracy studies , 2016, Statistics in medicine.

[34]  C. Black,et al.  Evaluation of Nucleic Acid Amplification Tests as Reference Tests for Chlamydia trachomatis Infections in Asymptomatic Men , 2000, Journal of Clinical Microbiology.

[35]  Wesley O Johnson,et al.  Identifiability of Models for Multiple Diagnostic Testing in the Absence of a Gold Standard , 2010, Biometrics.

[36]  Kate E. Jones,et al.  Global trends in emerging infectious diseases , 2008, Nature.

[37]  M. van Smeden,et al.  Diagnostic research in the absence of a gold standard , 2016 .

[38]  W. Miller,et al.  Bias in discrepant analysis: when two wrongs don't make a right. , 1998, Journal of clinical epidemiology.

[39]  S. Merhar,et al.  Letter to the editor , 2005, IEEE Communications Magazine.

[40]  A. P. Dawid,et al.  Maximum Likelihood Estimation of Observer Error‐Rates Using the EM Algorithm , 1979 .