A re‐evaluation of fixed effect(s) meta‐analysis

Meta†analysis is a common tool for synthesizing results of multiple studies. Among methods for performing meta†analysis, the approach known as ‘fixed effects’ or ‘inverse variance weighting’ is popular and widely used. A common interpretation of this method is that it assumes that the underlying effects in contributing studies are identical, and for this reason it is sometimes dismissed by practitioners. However, other interpretations of fixed effects analyses do not make this assumption, yet appear to be little known in the literature. We review these alternative interpretations, describing both their strengths and their limitations. We also describe how heterogeneity of the underlying effects can be addressed, with the same minimal assumptions, through either testing or meta†regression. Recommendations for the practice of meta†analysis are given; it is hoped that these will foster more direct connection of the questions that meta†analysts wish to answer with the statistical methods they choose.

[1]  Junichi Sakamoto,et al.  Individual patient-level and study-level meta-analysis for investigating modifiers of treatment effect. , 2004, Japanese journal of clinical oncology.

[2]  Day Ne,et al.  Testing hypotheses in case-control studies--equivalence of Mantel-Haenszel statistics and logit score tests. , 1979 .

[3]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[4]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[5]  Robert A Parker,et al.  BMC Medical Research Methodology BioMed Central BMC 22002, Medical Research Methodology , 2001 .

[6]  D. R. Cox,et al.  Interpretation of interaction: A review , 2007, 0712.1106.

[7]  Avshalom Caspi,et al.  Gene–environment interactions in psychiatry: joining forces with neuroscience , 2006, Nature Reviews Neuroscience.

[8]  John P A Ioannidis,et al.  Discovery properties of genome-wide association signals from cumulatively combined data sets. , 2009, American journal of epidemiology.

[9]  A Donner,et al.  The large-sample relative efficiency of the Mantel-Haenszel estimator in the fixed-strata case. , 1986, Biometrics.

[10]  A. Robinson Applied Asymptotics: Case Studies in Small-sample Statistics , 2008 .

[11]  J. Fleiss Review papers : The statistical basis of meta-analysis , 1993 .

[12]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[13]  B. M. Pötscher,et al.  MODEL SELECTION AND INFERENCE: FACTS AND FICTION , 2005, Econometric Theory.

[14]  Harold I Feldman,et al.  Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head , 2002, Statistics in medicine.

[15]  Manuel A. R. Ferreira,et al.  Practical aspects of imputation-driven meta-analysis of genome-wide association studies. , 2008, Human molecular genetics.

[16]  Ammarin Thakkinstian,et al.  Methods for meta-analyses of genome-wide association studies: critical assessment of empirical evidence. , 2012, American journal of epidemiology.

[17]  Norman E. Breslow,et al.  Odds ratio estimators when the data are sparse , 1981 .

[18]  D. Zeng,et al.  On the relative efficiency of using summary statistics versus individual-level data in meta-analysis. , 2010, Biometrika.

[19]  L. Hedges,et al.  The power of statistical tests in meta-analysis. , 2001, Psychological methods.

[20]  P. J. Huber The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[21]  Sonja M. McKINLAY,et al.  The effect of nonzero second-order interaction on combined estimators of the odds ratio , 1978 .

[22]  D. Spiegelhalter,et al.  Disease Mapping With WinBUGS and MLwiN, Bayesian Approaches to Clinical Trials and Health Care Evaluation , 2004 .

[23]  Nicky J Welton,et al.  Multiparameter evidence synthesis in epidemiology and medical decision-making , 2008, Journal of health services research & policy.

[24]  R. Prentice,et al.  Correcting “winner's curse” in odds ratios from genomewide association findings for major complex human diseases , 2009, Genetic epidemiology.

[25]  Julian P T Higgins,et al.  Controlling the risk of spurious findings from meta‐regression , 2004, Statistics in medicine.

[26]  Jonathan J Shuster,et al.  Empirical vs natural weighting in random effects meta‐analysis , 2009, Statistics in medicine.

[27]  Victoria Martínez,et al.  Introduction to Meta-Analysis , 2002 .

[28]  Gerta Rücker,et al.  Comments on 'Empirical vs natural weighting in random effects meta-analysis' by JJ Shuster, Statistics in Medicine 2009; 26, Published online, DOI: 10.1002/sim.3607. , 2010, Statistics in medicine.

[29]  Andrew D. Johnson,et al.  Genome-wide association study of blood pressure and hypertension , 2009, Nature Genetics.

[30]  Thomas Lumley,et al.  Complex Surveys: A Guide to Analysis Using R , 2010 .

[31]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[32]  S G Thompson,et al.  Controversies in meta-analysis: the case of the trials of serum cholesterol reduction , 1993, Statistical methods in medical research.

[33]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[34]  M. Stephens,et al.  Bayesian statistical methods for genetic association studies , 2009, Nature Reviews Genetics.

[35]  K. Taylor,et al.  Genome-Wide Association , 2007, Diabetes.

[36]  D. Siegmund Sequential Analysis: Tests and Confidence Intervals , 1985 .

[37]  Mike Clarke,et al.  Polychemotherapy for early breast cancer: an overview of the randomised trials , 1998, The Lancet.

[38]  P. Onghena,et al.  Parametric and nonparametric bootstrap methods for meta-analysis , 2005, Behavior research methods.

[39]  Andreas Buja,et al.  Models as Approximations - A Conspiracy of Random Regressors and Model Deviations Against Classical Inference in Regression , 2015 .

[40]  R Peto,et al.  Why do we need systematic overviews of randomized trials? , 1987, Statistics in medicine.

[41]  Mike Clarke,et al.  Tamoxifen for early breast cancer: an overview of the randomised trials , 1998, The Lancet.

[42]  A Whitehead,et al.  Meta‐analysis of continuous outcome data from individual patients , 2001, Statistics in medicine.

[43]  Stephen Senn,et al.  (www.interscience.wiley.com) DOI: 10.1002/sim.2639 Trying to be precise about vagueness , 2022 .

[44]  Joel A Waksman Comments on 'Empirical vs natural weighting in random effects meta-analysis'. , 2010, Statistics in medicine.

[45]  Julian P T Higgins,et al.  Graphical displays for meta‐analysis: An overview with suggestions for practice , 2010, Research synthesis methods.

[46]  David C Hoaglin,et al.  Misunderstandings about Q and ‘Cochran's Q test' in meta‐analysis , 2016, Statistics in medicine.

[47]  Eugene Demidenko,et al.  Multivariate meta-analysis for data consortia, individual patient meta-analysis, and pooling projects , 2008 .

[48]  J. R. Landis,et al.  A general overview of Mantel-Haenszel methods: applications and recent developments. , 1988, Annual review of public health.

[49]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[50]  Rainer Breitling,et al.  RankProd: a bioconductor package for detecting differentially expressed genes in meta-analysis , 2006, Bioinform..

[51]  H. White A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[52]  W. Haenszel,et al.  Statistical aspects of the analysis of data from retrospective studies of disease. , 1959, Journal of the National Cancer Institute.

[53]  I Olkin,et al.  Comparison of meta-analysis versus analysis of variance of individual patient data. , 1998, Biometrics.

[54]  M. Berman A theorem of Jacobi and its generalization , 1988 .

[55]  A Whitehead,et al.  A general parametric approach to the meta-analysis of randomized clinical trials. , 1991, Statistics in medicine.

[56]  Elena Kulinskaya,et al.  Testing for Homogeneity in Meta‐Analysis I. The One‐Parameter Case: Standardized Mean Difference , 2009, Biometrics.

[57]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[58]  Jean Meloche,et al.  Statistical Aspects of the Analysis of Data Networks , 2007, Technometrics.

[59]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[60]  Larry V. Hedges,et al.  Analyzing effect sizes: Fixed-effects models , 2009 .

[61]  David C Hoaglin,et al.  We know less than we should about methods of meta‐analysis , 2015, Research synthesis methods.

[62]  David J Spiegelhalter,et al.  Bias modelling in evidence synthesis , 2009, Journal of the Royal Statistical Society. Series A,.

[63]  M Egger,et al.  Intravenous magnesium for acute myocardial infarction. , 2007, The Cochrane database of systematic reviews.

[64]  D. Spiegelhalter,et al.  Bayesian Approaches to Clinical Trials and Health-Care Evaluation: Spiegelhalter/Clinical Trials and Health-Care Evaluation , 2004 .

[65]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[66]  S Greenland,et al.  Interpretation and estimation of summary ratios under heterogeneity. , 1982, Statistics in medicine.

[67]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[68]  Annamaria Guolo,et al.  Higher‐order likelihood inference in meta‐analysis and meta‐regression , 2012, Statistics in medicine.

[69]  Takemi Yanagimoto,et al.  Combining moment estimates of a parameter common through strata , 1990 .

[70]  D. Rubin Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation? , 1992 .

[71]  Elena Kulinskaya,et al.  On the moments of Cochran's Q statistic under the null hypothesis, with application to the meta‐analysis of risk difference , 2011, Research synthesis methods.

[72]  Alessandra Rosalba Brazzale hoa: An R Package Bundle for Higher Order Likelihood Inference , 2005 .

[73]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[74]  Theo Stijnen,et al.  Advanced methods in meta‐analysis: multivariate approach and meta‐regression , 2002, Statistics in medicine.

[75]  N. Day,et al.  Testing hypotheses in case-control studies--equivalence of Mantel-Haenszel statistics and logit score tests. , 1979, Biometrics.

[76]  B. Sinha,et al.  Statistical Meta-Analysis with Applications , 2008 .

[77]  C S Berkey,et al.  A random-effects regression model for meta-analysis. , 1995, Statistics in medicine.

[78]  Patrick Bossuyt,et al.  Systematic Reviews of Diagnostic Test Accuracy , 2008, Annals of Internal Medicine.

[79]  Robert J. Brooker,et al.  Genetics: Analysis and Principles , 1998 .

[80]  Allan Donner,et al.  The asymptotic relative efficiency of the Mantel-Haenszel estimator in the increasing-number-of-strata case , 1988 .

[81]  M. Schervish Theory of Statistics , 1995 .

[82]  N. Breslow,et al.  Statistics in Epidemiology : The Case-Control Study , 2008 .