An Empirical Comparison of Meta- and Mega-Analysis With Data From the ENIGMA Obsessive-Compulsive Disorder Working Group

Objective: Brain imaging communities focusing on different diseases have increasingly started to collaborate and to pool data to perform well-powered meta- and mega-analyses. Some methodologists claim that a one-stage individual-participant data (IPD) mega-analysis can be superior to a two-stage aggregated data meta-analysis, since more detailed computations can be performed in a mega-analysis. Before definitive conclusions regarding the performance of either method can be drawn, it is necessary to critically evaluate the methodology of, and results obtained by, meta- and mega-analyses. Methods: Here, we compare the inverse variance weighted random-effect meta-analysis model with a multiple linear regression mega-analysis model, as well as with a linear mixed-effects random-intercept mega-analysis model, using data from 38 cohorts including 3,665 participants of the ENIGMA-OCD consortium. We assessed the effect sizes and standard errors, and the fit of the models, to evaluate the performance of the different methods. Results: The mega-analytical models showed lower standard errors and narrower confidence intervals than the meta-analysis. Similar standard errors and confidence intervals were found for the linear regression and linear mixed-effects random-intercept models. Moreover, the linear mixed-effects random-intercept models showed better fit indices compared to linear regression mega-analytical models. Conclusions: Our findings indicate that results obtained by meta- and mega-analysis differ, in favor of the latter. In multi-center studies with a moderate amount of variation between cohorts, a linear mixed-effects random-intercept mega-analytical framework appears to be the better approach to investigate structural neuroimaging data.

Dan J. Stein | Paul M. Thompson | Patricia Gruner | Zhen Wang | Neda Jahanshad | Yuki Sakai | Derrek P. Hibar | Marcelo Queiroz Hoexter | Rachel Marsh | Ganesan Venkatasubramanian | Luciano Minuzzi | Guido A. van Wingen | Jun Soo Kwon | Michael C. Stevens | Christian Kaufmann | Philip R. Szeszko | Lianne Schmaal | David F. Tolin | Odile A. van den Heuvel | Alan Anticevic | Anthony James | John Piacentini | Gianfranco Spalletta | Fabrizio Piras | José M. Menchón | Carles Soriano-Mas | Hao Hu | Damiaan Denys | Martijn W. Heymans | João R. Sato | Jean-Paul Fouche | Tomohiro Nakao | H. Blair Simpson | David Mataix-Cols | Takashi Nakamae | Jan Carl Beucke | Jos W. R. Twisk | Janardhanan C. Narayanaswamy | Silvia Brem | Ignacio Martínez-Zalacaín | Je-Yeon Yun | Christine Lochner | Noam Soreni | Federica Piras | Jamie D. Feusner | Premika S. W. Boedhoe | Yoshinari Abe | Pino Alonso | Stephanie H. Ameis | Paul D. Arnold | Marcelo C. Batistuzzo | Francesco Benedetti | Irene Bollettini | Anushree Bose | Anna Calvo | Rosa Calvo | Yuqi Cheng | Kang Ik K. Cho | Valentina Ciullo | Sara Dallaspezia | Kate D. Fitzgerald | Egill A. Fridgeirsson | Gregory L. Hanna | Chaim Huyser | Norbert Kathmann | Kathrin Koch | Luisa Lazaro | Astrid Morer | Seiji Nishida | Erika L. Nurmi | Joseph O'Neill | Y. C. Janardhan Reddy | Tim J. Reess | Susanne Walitza | ENIGMA-OCD Working-Group | N. Jahanshad | P. Thompson | D. Stein | P. Szeszko | A. James | J. Kwon | F. Piras | F. Piras | G. Spalletta | A. Anticevic | D. Hibar | T. Nakamae | R. Calvo | L. Lázaro | M. Stevens | K. Koch | J. Twisk | N. Kathmann | L. Minuzzi | R. Marsh | C. Kaufmann | M. Heymans | D. Denys | C. Soriano-Mas | J. O’Neill | J. Sato | S. Ameis | H. Simpson | S. Walitza | S. Brem | G. Hanna | D. Tolin | C. Huyser | O. Heuvel | L. Schmaal | P. Alonso | J. Menchón | P. Boedhoe | J. Fouche | Y. Sakai | Y. Abe | P. Arnold | M. Batistuzzo | F. Benedetti | J. Beucke | I. Bollettini | A. Bose | A. Calvo | Yuqi Cheng | K. Cho | S. Dallaspezia | K. Fitzgerald | P. Gruner | M. Hoexter | C. Lochner | I. Martínez-Zalacaín | D. Mataix-Cols | T. Nakao | J. Narayanaswamy | Y. Reddy | G. Venkatasubramanian | Zhen Wang | J. Yun | E. Fridgeirsson | A. Morer | E. Nurmi | J. Piacentini | S. Nishida | V. Ciullo | T. Reess | G. Wingen | Hao Hu | N. Soreni | ENIGMA-OCD Working-Group | Tomohiro Nakao | P. Thompson | Seiji Nishida | P. Thompson | P. Thompson | P. Thompson

[1]  A. Agresti,et al.  Strategies for comparing treatments on a binary response with multi-centre data. , 2000, Statistics in medicine.

[2]  Anders M. Dale,et al.  Differentiating maturational and aging-related changes of the cerebral cortex by use of thickness and signal intensity , 2010, NeuroImage.

[3]  Andrea Benedetti,et al.  Systematic review of methods for individual patient data meta- analysis with binary outcomes , 2014, BMC Medical Research Methodology.

[4]  Manuel A. R. Ferreira,et al.  Practical aspects of imputation-driven meta-analysis of genome-wide association studies. , 2008, Human molecular genetics.

[5]  Judy Payne,et al.  Association and Causation , 2004 .

[6]  B. Kahan,et al.  Improper analysis of trials randomised using stratified blocks or minimisation , 2012, Statistics in medicine.

[7]  Richard D Riley,et al.  Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ , 2016, Statistics in medicine.

[8]  T. Mathew,et al.  On the Equivalence of Meta‐Analysis Using Literature and Using Individual Patient Data , 1999, Biometrics.

[9]  Silvia Brem,et al.  Cortical Abnormalities Associated With Pediatric and Adult Obsessive-Compulsive Disorder: Findings From the ENIGMA Obsessive-Compulsive Disorder Working Group. , 2017, The American journal of psychiatry.

[10]  J B Woodward,et al.  The Functional Magnetic Resonance Imaging Data Center (fMRIDC): the challenges and rewards of large-scale databasing of neuroimaging studies. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[11]  I Olkin,et al.  Comparison of meta-analysis versus analysis of variance of individual patient data. , 1998, Biometrics.

[12]  Thomas E. Nichols,et al.  The ENIGMA Consortium: large-scale collaborative analyses of neuroimaging and genetic data , 2014, Brain Imaging and Behavior.

[13]  Bruce Fischl,et al.  FreeSurfer , 2012, NeuroImage.

[14]  L. Stewart,et al.  Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. , 1995, Statistics in medicine.

[15]  Thomas Mathew,et al.  Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data , 2010, Biometrical journal. Biometrische Zeitschrift.

[16]  Alan C. Evans,et al.  Brain size and cortical structure in the adult human brain. , 2008, Cerebral cortex.

[17]  Wiro J Niessen,et al.  Common variants at 6q22 and 17q21 are associated with intracranial volume , 2012, Nature Genetics.

[18]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[19]  Matthew Dutton,et al.  Individual Patient-Level Data meta-analysis: A comparison of methods for the Diverse Populations Collaboration data set , 2011 .

[20]  D Y Lin,et al.  Meta‐analysis of genome‐wide association studies: no efficiency gain in using individual participant data , 2009, Genetic epidemiology.

[21]  L. Stewart,et al.  To IPD or not to IPD? , 2002, Evaluation & the health professions.

[22]  Wiro J. Niessen,et al.  Erratum: Common variants at 6q22 and 17q21 are associated with intracranial volume (Nature Genetics (2012) 44 (539-544)) , 2013 .

[23]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[24]  Sébastien Ourselin,et al.  Head size, age and gender adjustment in MRI studies: a necessary nuisance? , 2010, NeuroImage.

[25]  Hannah R Rothstein,et al.  A basic introduction to fixed‐effect and random‐effects models for meta‐analysis , 2010, Research synthesis methods.

[26]  D. Harville Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems , 1977 .

[27]  S. Stewart,et al.  Association and Causation in Brain Imaging in the Case of OCD : Response to McKay et al , 2017 .

[28]  I. Cuthill,et al.  Effect size, confidence interval and statistical significance: a practical guide for biologists , 2007, Biological reviews of the Cambridge Philosophical Society.

[29]  D. Zeng,et al.  On the relative efficiency of using summary statistics versus individual-level data in meta-analysis. , 2010, Biometrika.

[30]  Silvia Brem,et al.  Distinct Subcortical Volume Alterations in Pediatric and Adult OCD: A Worldwide Meta- and Mega-Analysis. , 2017, The American journal of psychiatry.