The power of statistical tests in meta-analysis.

Calculations of the power of statistical tests are important in planning research studies (including meta-analyses) and in interpreting situations in which a result has not proven to be statistically significant. The authors describe procedures to compute statistical power of fixed- and random-effects tests of the mean effect size, tests for heterogeneity (or variation) of effect size parameters across studies, and tests for contrasts among effect sizes of different studies. Examples are given using 2 published meta-analyses. The examples illustrate that statistical power is not always high in meta-analysis.

[1]  L. Hedges.  Fitting Categorical Models to Effect Sizes from a Series of Experiments , 1982 .

[2]  John E. Hunter,et al.  Development of a general solution to the problem of validity generalization. , 1977 .

[3]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[4]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[5]  D. Rubin,et al.  Comparing Effect Sizes of Independent Studies , 1982 .

[6]  Stephen Dubin.  How many subjects? Statistical power analysis in research , 1990 .

[7]  Satterthwaite Fe.  An approximate distribution of estimates of variance components. , 1946 .

[8]  John E. Hunter,et al.  Validity generalization results for computer programmers. , 1980 .

[9]  Roel Bosker,et al.  Standard Errors and Sample Sizes for Two-Level Research , 1993 .

[10]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data , 1991 .

[11]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[12]  Larry V. Hedges,et al.  A random effects model for effect sizes , 1983 .

[13]  R. Bauserman,et al.  A meta-analytic examination of assumed properties of child sexual abuse using college samples. , 1998, Psychological bulletin.

[14]  Mark W. Lipsey,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[15]  P. Diggle.  Analysis of Longitudinal Data , 1995 .

[16]  V. Pollock,et al.  The Long-Term Sequelae of Childhood Sexual Abuse in Women: A Meta-Analytic Review , 1996 .

[17]  Helena C. Kraemer,et al.  Advantages of excluding underpowered studies in meta-analysis: Inclusionist versus exclusionist viewpoints. , 1998 .

[18]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[19]  L. Hedges.  Estimation of effect size from a series of independent experiments. , 1982 .

[20]  P. Lachenbruch,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[21]  M. Lipsey,et al.  The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. , 1993, American Psychologist.

[22]  F. E. Satterthwaite.  An approximate distribution of estimates of variance components. , 1946, Biometrics.

[23]  Frank L. Schmidt,et al.  What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. , 1992 .

[24]  P. Patnaik.  The Non-central X^2- and F- distribution and Their Applications , 1949 .

[25]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data , 1992 .

[26]  D Finkelhor,et al.  Impact of sexual abuse on children: a review and synthesis of recent empirical studies. , 1993, Psychological bulletin.