Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models

BackgroundMeta-regression is becoming increasingly used to model study level covariate effects. However this type of statistical analysis presents many difficulties and challenges. Here two methods for calculating confidence intervals for the magnitude of the residual between-study variance in random effects meta-regression models are developed. A further suggestion for calculating credible intervals using informative prior distributions for the residual between-study variance is presented.MethodsTwo recently proposed and, under the assumptions of the random effects model, exact methods for constructing confidence intervals for the between-study variance in random effects meta-analyses are extended to the meta-regression setting. The use of Generalised Cochran heterogeneity statistics is extended to the meta-regression setting and a Newton-Raphson procedure is developed to implement the Q profile method for meta-analysis and meta-regression. WinBUGS is used to implement informative priors for the residual between-study variance in the context of Bayesian meta-regressions.ResultsResults are obtained for two contrasting examples, where the first example involves a binary covariate and the second involves a continuous covariate. Intervals for the residual between-study variance are wide for both examples.ConclusionsStatistical methods, and R computer software, are available to compute exact confidence intervals for the residual between-study variance under the random effects model for meta-regression. These frequentist methods are almost as easily implemented as their established counterparts for meta-analysis. Bayesian meta-regressions are also easily performed by analysts who are comfortable using WinBUGS. Estimates of the residual between-study variance in random effects meta-regressions should be routinely reported and accompanied by some measure of their uncertainty. Confidence and/or credible intervals are well-suited to this purpose.

[1]  Dan Jackson,et al.  A design-by-treatment interaction model for network meta-analysis with random inconsistency effects , 2014, Statistics in medicine.

[2]  Dan Jackson,et al.  Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression‡ , 2012, Research synthesis methods.

[3]  Simon G Thompson,et al.  Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews , 2012, International journal of epidemiology.

[4]  David R. Jones,et al.  How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS , 2005, Statistics in medicine.

[5]  Dan Jackson,et al.  The exact distribution of Cochran's heterogeneity statistic in one‐way random effects meta‐analysis , 2008, Statistics in medicine.

[6]  Eleanor M Pullenayegum,et al.  An informed reference prior for between‐study heterogeneity in meta‐analyses of binary outcomes , 2011, Statistics in medicine.

[7]  M. Thorogood,et al.  Interventions for promoting physical activity. , 2005, The Cochrane database of systematic reviews.

[8]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[9]  D. Bereczki,et al.  The Cochrane Database of Systematic Reviews , 2003 .

[10]  H. Ahrens.  Searle, S. R.: Linear Models. John Wiley & Sons, Inc., New York-London-Sydney-Toronto 1971. XXI, 532 S. $9.50 , 1974 .

[11]  W. Viechtbauer.  Analysis of moderator effects in meta-analysis , 2008 .

[12]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1999 .

[13]  J. Cappelleri,et al.  Understanding heterogeneity in meta‐analysis: the role of meta‐regression , 2009, International journal of clinical practice.

[14]  Dan Jackson.  The Significance Level of Meta-Regression's Standard Hypothesis Test , 2008 .

[15]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[16]  Dan Jackson,et al.  Inference for meta-analysis with a suspected temporal trend. , 2010, Biometrical journal. Biometrische Zeitschrift.

[17]  Theo Stijnen,et al.  Advanced methods in meta‐analysis: multivariate approach and meta‐regression , 2002, Statistics in medicine.

[18]  Raghu Kacker,et al.  Random-effects model for meta-analysis of clinical trials: an update. , 2007, Contemporary clinical trials.

[19]  John Mandel,et al.  Consensus Values and Weighting Factors. , 1982, Journal of research of the National Bureau of Standards.

[20]  Guido Knapp,et al.  On Estimating Residual Heterogeneity in Random-Effects Meta-Regression: A Comparative Study , 2013, J. Stat. Theory Appl..

[21]  Kristian Thorlund,et al.  Modelling heterogeneity variances in multiple treatment comparison meta-analysis – Are informative priors the better solution? , 2013, BMC Medical Research Methodology.

[22]  Rose Baker,et al.  Meta‐analysis inside and outside particle physics: convergence using the path of least resistance? , 2013, Research synthesis methods.

[23]  P. Coulthard,et al.  Paracetamol for pain relief after surgical removal of lower wisdom teeth. , 2007, The Cochrane database of systematic reviews.

[24]  Dan Jackson,et al.  Confidence intervals for the between‐study variance in random effects meta‐analysis using generalised Cochran heterogeneity statistics , 2013, Research synthesis methods.

[25]  J. Sánchez-Meca,et al.  A comparison of procedures to test for moderators in mixed-effects meta-regression models. , 2015, Psychological methods.

[26]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.

[27]  B J Biggerstaff,et al.  Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. , 1997, Statistics in medicine.

[28]  R. Farebrother.  The Distribution of a Positive Linear Combination of X2 Random Variables , 1984 .

[29]  Guobing Lu,et al.  Modeling between-trial variance structure in mixed treatment comparisons. , 2009, Biostatistics.

[30]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1997, Statistics in medicine.

[31]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[32]  Julian P T Higgins,et al.  Comparative efficacy and safety of treatments for localised prostate cancer: an application of network meta-analysis , 2014, BMJ Open.

[33]  Wolfgang Viechtbauer,et al.  Confidence intervals for the amount of heterogeneity in meta‐analysis , 2007, Statistics in medicine.

[34]  S. R. Searle.  Linear Models , 1971 .

[35]  Dan Jackson,et al.  Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis , 2014, Statistics in medicine.

[36]  Guido Knapp,et al.  Improved tests for a random effects meta‐regression with a single covariate , 2003, Statistics in medicine.

[37]  Guido Knapp,et al.  On confidence intervals for the among-group variance in the one-way random effects model with unequal error variances , 2005 .

[38]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[39]  J. Osborne.  Best Practices in Quantitative Methods , 2009 .

[40]  Nicky J Welton,et al.  Linear inference for mixed treatment comparison meta‐analysis: A two‐stage approach , 2011, Research synthesis methods.

[41]  C S Berkey,et al.  A random-effects regression model for meta-analysis. , 1995, Statistics in medicine.

[42]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[43]  Guido Knapp,et al.  Assessing the Amount of Heterogeneity in Random‐Effects Meta‐Analysis , 2006, Biometrical journal. Biometrische Zeitschrift.