Meta-analysis of non-statistically significant unreported effects

Published studies in Medicine (and virtually any other discipline) sometimes report that a difference or correlation did not reach statistical significance but do not report its effect size or any statistic from which the latter may be derived. Unfortunately, meta-analysts should not exclude these studies because their exclusion would bias the meta-analytic outcome, but also they cannot be included as null effect sizes because this strategy is also associated to bias. To overcome this problem, we have developed MetaNSUE, a novel method based on multiple imputations of the censored information. We also provide an R package and an easy-to-use Graphical User Interface for non-R meta-analysts.

[1]  Ralf Bender,et al.  Methods to estimate the between‐study variance and its uncertainty in meta‐analysis† , 2015, Research synthesis methods.

[2]  Roger A. Sugden,et al.  Multiple Imputation for Nonresponse in Surveys , 1988 .

[3]  Wendelin Schnedler,et al.  Likelihood Estimation for Censored Random Vectors , 2005 .

[4]  H. Hart,et al.  Structural and Functional Brain Abnormalities in Attention-Deficit/Hyperactivity Disorder and Obsessive-Compulsive Disorder: A Comparative Meta-analysis. , 2016, JAMA psychiatry.

[5]  Stephen Burgess,et al.  Combining multiple imputation and meta-analysis with individual participant data , 2013, Statistics in medicine.

[6]  J. Raduà,et al.  Meta-Analysis of fMRI Studies of Disruptive Behavior Disorders. , 2016, The American journal of psychiatry.

[7]  Wolfgang Viechtbauer,et al.  Bias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model , 2005 .

[8]  M. Brammer,et al.  Effects of Stimulants on Brain Function in Attention-Deficit/Hyperactivity Disorder: A Systematic Review and Meta-Analysis , 2014, Biological Psychiatry.

[9]  J. Fox,et al.  Applied Regression Analysis and Generalized Linear Models , 2008 .

[10]  J Radua,et al.  A new meta-analytic method for neuroimaging studies that combines reported peak coordinates and statistical parametric maps , 2012, European Psychiatry.

[11]  John Fox,et al.  Applied Regression Analysis and Generalized Linear Models , 2008 .

[12]  J. Tobin Estimation of Relationships for Limited Dependent Variables , 1958 .

[13]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[14]  P. McGuire,et al.  Ventral Striatal Activation During Reward Processing in Psychosis: A Neurofunctional Meta-Analysis. , 2015, JAMA psychiatry.

[15]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[16]  Donald B. Rubin,et al.  Significance levels from repeated p-values with multiply imputed data , 1991 .