An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary.

OBJECTIVE Meta-analysis of studies of the accuracy of diagnostic tests currently uses a variety of methods. Statistically rigorous hierarchical models require expertise and sophisticated software. We assessed whether any of the simpler methods can in practice give adequately accurate and reliable results. STUDY DESIGN AND SETTING We reviewed six methods for meta-analysis of diagnostic accuracy: four simple commonly used methods (simple pooling, separate random-effects meta-analyses of sensitivity and specificity, separate meta-analyses of positive and negative likelihood ratios, and the Littenberg-Moses summary receiver operating characteristic [ROC] curve) and two more statistically rigorous approaches using hierarchical models (bivariate random-effects meta-analysis and hierarchical summary ROC curve analysis). We applied the methods to data from a sample of eight systematic reviews chosen to illustrate a variety of patterns of results. RESULTS In each meta-analysis, there was substantial heterogeneity between the results of different studies. Simple pooling of results gave misleading summary estimates of sensitivity and specificity in some meta-analyses, and the Littenberg-Moses method produced summary ROC curves that diverged from those produced by more rigorous methods in some situations. CONCLUSION The closely related hierarchical summary ROC curve or bivariate models should be used as the standard method for meta-analysis of diagnostic accuracy.

[1]  J. Steurer,et al.  Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review , 2003, BMJ : British Medical Journal.

[2]  David J Samson,et al.  Challenges in Systematic Reviews of Diagnostic Technologies , 2005, Annals of Internal Medicine.

[3]  Richard D Riley,et al.  Beyond the Bench: Hunting Down Fugitive Literature , 2004, Environmental Health Perspectives.

[4]  Douglas G. Altman,et al.  Systematic Reviews in Health Care , 2001 .

[5]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. , 2004, Family practice.

[6]  Lucas M Bachmann,et al.  Accuracy of cervicovaginal fetal fibronectin test in predicting risk of spontaneous preterm birth: systematic review , 2002, BMJ : British Medical Journal.

[7]  C Gatsonis,et al.  Meta‐analysis of Diagnostic Test Accuracy Assessment Studies with Varying Number of Thresholds , 2003, Biometrics.

[8]  L E Moses,et al.  Estimating Diagnostic Accuracy from Multiple Conflicting Reports , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[9]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[10]  P Glasziou,et al.  Meta-analytic methods for diagnostic test accuracy. , 1995, Journal of clinical epidemiology.

[11]  P. Bossuyt,et al.  Sources of Variation and Bias in Studies of Diagnostic Accuracy , 2004, Annals of Internal Medicine.

[12]  O S Miettinen,et al.  The modern scientific physician: 3. Scientific diagnosis. , 2001, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[13]  N. Choudhry,et al.  The conditional relative odds ratio provided less biased results for comparing diagnostic test accuracy in meta-analyses. , 2004, Journal of clinical epidemiology.

[14]  Lucas M Bachmann,et al.  Accuracy of electrocardiography in diagnosis of left ventricular hypertrophy in arterial hypertension: systematic review , 2007, BMJ : British Medical Journal.

[15]  L E Moses,et al.  Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. , 1993, Statistics in medicine.

[16]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[17]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[18]  J. Knottnerus,et al.  Assessment of the accuracy of diagnostic tests: the cross-sectional study. , 2003, Journal of clinical epidemiology.

[19]  Christian Gluud,et al.  Evidence based diagnostics , 2005, BMJ : British Medical Journal.

[20]  P C Lambert,et al.  An evaluation of bivariate random‐effects meta‐analysis for the joint synthesis of two correlated outcomes , 2007, Statistics in medicine.

[21]  S D Walter,et al.  Studies reporting ROC curves of diagnostic and prediction data can be incorporated into meta-analyses using corresponding odds ratios. , 2007, Journal of clinical epidemiology.

[22]  Haitao Chu,et al.  Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. , 2006, Journal of clinical epidemiology.

[23]  Roger M Harbord,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2007, Biostatistics.

[24]  Jonathan J Deeks,et al.  Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. , 2001, BMJ.

[25]  Paul Glasziou,et al.  Comparative accuracy: assessing new tests against existing diagnostic pathways , 2006, BMJ : British Medical Journal.

[26]  Diederick E Grobbee,et al.  Test research versus diagnostic research. , 2004, Clinical chemistry.

[27]  Julian P T Higgins,et al.  Controlling the risk of spurious findings from meta‐regression , 2004, Statistics in medicine.

[28]  Jos Kleijnen,et al.  The accuracy of various tests for bacterial vaginosis in predicting preterm birth: a systematic review , 2004 .

[29]  Lucas M Bachmann,et al.  Accuracy of B-type natriuretic peptide tests to exclude congestive heart failure: systematic review of test accuracy studies. , 2006, Archives of internal medicine.

[30]  J. Deeks,et al.  A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. , 2005, Health technology assessment.

[31]  Victor M Montori,et al.  Conducting systematic reviews of diagnostic studies: didactic guidelines , 2002, BMC medical research methodology.

[32]  Lucas M Bachmann,et al.  Systematic reviews with individual patient data meta-analysis to evaluate diagnostic tests. , 2003, European journal of obstetrics, gynecology, and reproductive biology.

[33]  Javier Zamora,et al.  Meta-DiSc : a software for meta-analysis of test accuracy data , 2015 .

[34]  Petra Macaskill,et al.  Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. , 2004, Journal of clinical epidemiology.