What inferences can and cannot be made on the basis of meta-analysis?

Abstract Meta-analysis involves inferences about the mean, the variability and the substantive reasons for variability in effect sizes. The risks inherent in each of these inferences are reviewed, and a Bayesian approach to using meta-analysis to determine whether effects vary in important ways is suggested.

[1]  K. Murphy,et al.  Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests, Second Ediction , 1998 .

[2]  Patrick H. Raymark,et al.  The criterion-related validity of integrity tests: an updated meta-analysis. , 2012, The Journal of applied psychology.

[3]  Simon B Eickhoff,et al.  Meta-analysis in human neuroimaging: computational modeling of large-scale databases. , 2014, Annual review of neuroscience.

[4]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[5]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[6]  W. H. Stewart,et al.  Data quality affects meta-analytic conclusions: a response to Miner and Raju (2004) concerning entrepreneurial risk propensity. , 2004, The Journal of applied psychology.

[7]  Frederick L. Oswald,et al.  Meta-analysis and the art of the average. , 2003 .

[8]  Judith A. Hall,et al.  Testing for moderator variables in meta‐analysis: Issues and methods , 1991 .

[9]  M T Brannick.  Implications of empirical Bayes meta-analysis for test validation. , 2001, The Journal of applied psychology.

[10]  D. Rousseau,et al.  Evidence in Management and Organizational Science: Assembling the Field's Full Weight of Scientific Knowledge through Syntheses , 2008 .

[11]  T. A. Wright,et al.  Best-practice recommendations for estimating interaction effects using meta-analysis , 2011 .

[12]  M. Borenstein,et al.  Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments , 2006 .

[13]  R. Rosenthal,et al.  Meta-analysis: recent developments in quantitative methods for literature reviews. , 2001, Annual review of psychology.

[14]  Michael T Brannick,et al.  Comparison of two random-effects methods of meta-analysis. , 2002, The Journal of applied psychology.

[15]  Herman Aguinis,et al.  Meta-Analytic Choices and Judgment Calls: Implications for Theory Building and Testing, Obtained Effect Sizes, and Scholarly Impact , 2011 .

[16]  Piers Steel,et al.  Bayesian Variance Estimation for Meta-Analysis , 2008 .

[17]  F. Schmidt,et al.  History, Development, Evolution, and Impact of Validity Generalization and Meta-Analysis Methods, 1975–2001 , 2003 .

[18]  Steven M. Hall,et al.  Validity Generalization from a Bayesian Perspective , 2003 .

[19]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[20]  Frank L. Schmidt,et al.  What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. , 1992 .

[21]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[22]  Philip L. Roth,et al.  Meta-analysis and validity generalization as research tools: Issues of sample bias and degrees of mis-specification. , 2003 .

[23]  Larry V. Hedges,et al.  An unbiased correction for sampling error in validity generalization studies. , 1989 .

[24]  H. G. Osburn,et al.  Moderating effects of decision-making/information-processing job dimensions on test validities. , 1983 .

[25]  B. Thombs,et al.  Ain't necessarily so: review and critique of recent meta-analyses of behavioral medicine interventions in health psychology. , 2010, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[26]  Kevin D. Carlson,et al.  Citing and Building on Meta-Analytic Findings , 2011 .

[27]  Paul R. Sackett,et al.  On seeking moderator variables in the meta-analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error. , 1986 .

[28]  Herman Aguinis,et al.  Testing Moderator Variable Hypotheses Meta-Analytically: , 1998 .

[29]  James G. Field,et al.  Correlational effect size benchmarks. , 2015, The Journal of applied psychology.

[30]  Piers Steel,et al.  Improving the Meta-Analytic Assessment of Effect Size Variance With an Informed Bayesian Prior , 2015 .

[31]  F. Schmidt.  Meta-Analysis , 2008 .

[32]  Kathleen D. Vohs,et al.  Social Rejection, Control, Numbness, and Emotion: How Not to be Fooled by Gerber and Wheeler (2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[33]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[34]  In-Sue Oh,et al.  Fixed- versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results. , 2009, The British journal of mathematical and statistical psychology.

[35]  Kerrie Mengersen,et al.  Bayesian Meta‐Analysis , 2012 .

[36]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[37]  H. Eysenck.  An exercise in mega-silliness. , 1978 .

[38]  F. Schmidt,et al.  Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance. , 1993 .

[39]  Catherine M. Dalton,et al.  Debunking Myths and Urban Legends About Meta-Analysis , 2011 .

[40]  John P. Wanous,et al.  The role of judgment calls in meta-analysis. , 1989 .

[41]  Michael A. McDaniel,et al.  PUBLICATION BIAS: A CASE STUDY OF FOUR TEST VENDORS , 2006 .

[42]  Patricia B. Elmore,et al.  Correlational Meta-Analysis: Independent and Nonindependent Cases , 1992 .

[43]  B J Biggerstaff,et al.  Bayesian meta-analysis, with application to studies of ETS and lung cancer. , 1996, Lung cancer.

[44]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .