Guidance for deriving and presenting percentage study weights in meta‐analysis of test accuracy studies

Percentage study weights in meta-analysis reveal the contribution of each study toward the overall summary results and are especially important when some studies are considered outliers or at high risk of bias. In meta-analyses of test accuracy reviews, such as a bivariate meta-analysis of sensitivity and specificity, the percentage study weights are not currently derived. Rather, the focus is on representing the precision of study estimates on receiver operating characteristic plots by scaling the points relative to the study sample size or to their standard error. In this article, we recommend that researchers should also provide the percentage study weights directly, and we propose a method to derive them based on a decomposition of Fisher information matrix. This method also generalises to a bivariate meta-regression so that percentage study weights can also be derived for estimates of study-level modifiers of test accuracy. Application is made to two meta-analyses examining test accuracy: one of ear temperature for diagnosis of fever in children and the other of positron emission tomography for diagnosis of Alzheimer's disease. These highlight that the percentage study weights provide important information that is otherwise hidden if the presentation only focuses on precision based on sample size or standard errors. Software code is provided for Stata, and we suggest that our proposed percentage weights should be routinely added on forest and receiver operating characteristic plots for sensitivity and specificity, to provide transparency of the contribution of each study toward the results. This has implications for the PRISMA-diagnostic test accuracy guidelines that are currently being produced.

[1]  Evangelos Kontopantelis,et al.  A Short Guide and a Forest Plot Command (Ipdforest) for One-Stage Meta-Analysis , 2012 .

[2]  Susan Mallett,et al.  QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies , 2011, Annals of Internal Medicine.

[3]  Richard D Riley,et al.  A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression , 2013, Biometrical journal. Biometrische Zeitschrift.

[4]  Penny Whiting,et al.  Metandi: Meta-analysis of Diagnostic Accuracy Using Hierarchical Logistic Regression , 2009 .

[5]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.

[6]  Yemisi Takwoingi,et al.  Empirical Evidence of the Importance of Comparative Studies of Diagnostic Test Accuracy , 2013, Annals of Internal Medicine.

[7]  P. Williamson,et al.  In a systematic review, infrared ear thermometry for fever diagnosis in children finds poor sensitivity. , 2006, Journal of clinical epidemiology.

[8]  Haitao Chu,et al.  Statistical methods for multivariate meta-analysis of diagnostic tests: An overview and tutorial , 2016, Statistical methods in medical research.

[9]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[10]  Alex J Sutton,et al.  ‘Cross hairs’ plots for diagnostic meta‐analysis , 2010, Research synthesis methods.

[11]  Ian R. White,et al.  Multivariate Random-effects Meta-analysis , 2009 .

[12]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[13]  P Glasziou,et al.  Meta-analytic methods for diagnostic test accuracy. , 1995, Journal of clinical epidemiology.

[14]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[15]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[16]  J. Pinheiro,et al.  Efficient Laplacian and Adaptive Gaussian Quadrature Algorithms for Multilevel Generalized Linear Mixed Models , 2006 .

[17]  Paula R Williamson,et al.  Infrared ear thermometry compared with rectal thermometry in children: a systematic review , 2002, The Lancet.

[18]  Patrick M. M. Bossuyt,et al.  Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy , 2013 .

[19]  Richard D Riley,et al.  Meta‐analysis of diagnostic test studies using individual patient data and aggregate data , 2008, Statistics in medicine.

[20]  Haitao Chu,et al.  Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. , 2006, Journal of clinical epidemiology.

[21]  Roger M Harbord,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2007, Biostatistics.

[22]  H C Van Houwelingen,et al.  A bivariate approach to meta-analysis. , 1993, Statistics in medicine.

[23]  Richard D Riley,et al.  Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models , 2017, Statistical methods in medical research.