Effect Sizes in Cluster-Randomized Designs

Multisite research designs involving cluster randomization are becoming increasingly important in educational and behavioral research. Researchers would like to compute effect size indexes based on the standardized mean difference to compare the results of cluster-randomized studies (and corresponding quasi-experiments) with other studies and to combine information across studies in meta-analyses. This article addresses the problem of defining effect sizes in multilevel designs and computing estimates of those effect sizes and their standard errors from information that is likely to be reported in journal articles. Three effect sizes are defined corresponding to different standardizations. Estimators of each effect size index are also presented along with their sampling distributions (including standard errors).

[1]  L. Hedges,et al.  Intraclass Correlation Values for Planning Group-Randomized Trials in Education , 2007 .

[2]  J. Gill Hierarchical Linear Models , 2005 .

[3]  Jonathan L. Blitstein,et al.  Design and analysis of group-randomized trials: a review of recent methodological developments. , 2004, American journal of public health.

[4]  Denisse R. Thompson,et al.  Standards-based school mathematics curricula : What are they? What do students learn? , 2004 .

[5]  M. Laopaiboon,et al.  Meta-analyses involving cluster randomization trials: a review of published literature in health care , 2003, Statistical methods in medical research.

[6]  David M. Murray,et al.  Methods To Reduce The Impact Of Intraclass Correlation In Group-Randomized Trials , 2003, Evaluation review.

[7]  D. Sengupta Linear models , 2003 .

[8]  Karl J. Friston,et al.  Variance Components , 2003 .

[9]  Allan Donner,et al.  Issues in the meta‐analysis of cluster randomized trials , 2002, Statistics in medicine.

[10]  A Donner,et al.  Current and future challenges in the design and analysis of cluster randomization trials , 2001, Statistics in medicine.

[11]  Allan Donner,et al.  Design and Analysis of Cluster Randomization Trials in Health Research , 2001 .

[12]  S. Chinn,et al.  Components of variance and intraclass correlations for the design of community-based surveys and intervention studies: data from the Health Survey for England 1994. , 1999, American journal of epidemiology.

[13]  V. Verma,et al.  An analysis of sampling errors for the Demographic Health Surveys. , 1996, International statistical review = Revue internationale de statistique.

[14]  D M Murray,et al.  A Meta-Analysis of Smoking Prevention Programs After Adjustment for Errors in the Unit of Analysis , 1996, Health education quarterly.

[15]  A Donner,et al.  Randomization by cluster. , 1991, Nursing research.

[16]  A. Donner,et al.  Randomization by cluster. Sample size requirements and analysis. , 1981, American journal of epidemiology.

[17]  L. Hedges Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .

[18]  W. R. Buckland,et al.  Distributions in Statistics: Continuous Multivariate Distributions , 1973 .

[19]  P. R. Fisk,et al.  Distributions in Statistics: Continuous Multivariate Distributions , 1971 .

[20]  S. Geisser,et al.  An Extension of Box's Results on the Use of the $F$ Distribution in Multivariate Analysis , 1958 .

[21]  G. Box Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification , 1954 .

[22]  George W. Snedecor,et al.  Statistical methods applied to experiments in agriculture and biology. , 1946 .