Meta-Analyses of Diagnostic Accuracy in Imaging Journals: Analysis of Pooling Techniques and Their Effect on Summary Estimates of Diagnostic Accuracy.

Purpose To determine whether authors of systematic reviews of diagnostic accuracy studies published in imaging journals used recommended methods for meta-analysis, and to evaluate the effect of traditional methods on summary estimates of sensitivity and specificity. Materials and Methods Medline was searched for published systematic reviews that included meta-analysis of test accuracy data limited to imaging journals published from January 2005 to May 2015. Two reviewers independently extracted study data and classified methods for meta-analysis as traditional (univariate fixed- or random-effects pooling or summary receiver operating characteristic curve) or recommended (bivariate model or hierarchic summary receiver operating characteristic curve). Use of methods was analyzed for variation with time, geographical location, subspecialty, and journal. Results from reviews in which study authors used traditional univariate pooling methods were recalculated with a bivariate model. Results Three hundred reviews met the inclusion criteria, and in 118 (39%) of those, authors used recommended meta-analysis methods. No change in the method used was observed with time (r = 0.54, P = .09); however, there was geographic (χ(2) = 15.7, P = .001), subspecialty (χ(2) = 46.7, P < .001), and journal (χ(2) = 27.6, P < .001) heterogeneity. Fifty-one univariate random-effects meta-analyses were reanalyzed with the bivariate model; the average change in the summary estimate was -1.4% (P < .001) for sensitivity and -2.5% (P < .001) for specificity. The average change in width of the confidence interval was 7.7% (P < .001) for sensitivity and 9.9% (P ≤ .001) for specificity. Conclusion Recommended methods for meta-analysis of diagnostic accuracy in imaging journals are used in a minority of reviews; this has not changed significantly with time. Traditional (univariate) methods allow overestimation of diagnostic accuracy and provide narrower confidence intervals than do recommended (bivariate) methods. (©) RSNA, 2016 Online supplemental material is available for this article.

[1]  P. Bossuyt,et al.  Pitfalls of Systematic Reviews and Meta-Analyses in Imaging Research. , 2015, Radiology.

[2]  L. Hooft,et al.  Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study , 2014, BMC Medical Research Methodology.

[3]  Eleanor A. Ochodo,et al.  Survey revealed a lack of clarity about recommended methods for meta-analysis of diagnostic accuracy data. , 2013, Journal of clinical epidemiology.

[4]  Kaisra Esmail,et al.  Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? , 2013, Radiology.

[5]  T. Trikalinos,et al.  Survey of the methods and reporting practices in published meta‐analyses of test performance: 1987 to 2009 , 2013, Research synthesis methods.

[6]  Patrick M. M. Bossuyt,et al.  Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy , 2013 .

[7]  T. Trikalinos,et al.  Comprehensive Overview of Methods and Reporting of Meta-Analyses of Test Accuracy , 2012 .

[8]  Brian H Willis,et al.  Uptake of newer methodological developments and the deployment of meta-analysis in diagnostic test research: a systematic review , 2010, BMC medical research methodology.

[9]  Constantine Gatsonis,et al.  Analysing and Presenting Results , 2010 .

[10]  Patrick M M Bossuyt,et al.  Differences between univariate and bivariate models for summarizing diagnostic accuracy may not be large. , 2009, Journal of clinical epidemiology.

[11]  Haitao Chu,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2009, Biostatistics.

[12]  Patrick Bossuyt,et al.  Systematic Reviews of Diagnostic Test Accuracy , 2008, Annals of Internal Medicine.

[13]  Lucas M. Bachmann,et al.  An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary. , 2008, Journal of clinical epidemiology.

[14]  Roger M Harbord,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2007, Biostatistics.

[15]  Javier Zamora,et al.  Meta-DiSc : a software for meta-analysis of test accuracy data , 2015 .

[16]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[17]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[18]  P D Bezemer,et al.  Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. , 2000, Journal of clinical epidemiology.

[19]  L E Moses,et al.  Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. , 1993, Statistics in medicine.