Intraclass Correlations Values in International Development: Evidence Across Commonly Studied Domains in sub-Saharan Africa

The sharp increase in the number of experimental studies evaluating development programs raises the need for accurate intraclass correlations (ICC) to conduct power calculations so that researchers can design studies to detect meaningful effects with sufficient statistical power. The intraclass correlation is an important parameter for determining the statistical power of cluster-randomized trials. The parameter is rarely available to researchers planning a study until after the design is set and data are already collected. This paper takes an important step towards helping researchers working in sub-Saharan Africa to accurately estimate appropriate sample sizes for their clustered RCTs. The study draws from rich data sets in Kenya, Malawi, Zambia, and Zimbabwe. We present ICCs for a wide range of domains common for development research. Our results suggest that ICCs for commonly studied indicators in sub-Saharan Africa are lower than is often assumed in power calculations. ICC values are especially low for indicators associated with child nutrition and food security, suggesting that cluster-RCTs might be a viable design even when faced with limited budgets because sample size requirements are not much different from an individual random assignment design.

[1]  Susan Athey,et al.  When Should You Adjust Standard Errors for Clustering? , 2017, The Quarterly Journal of Economics.

[2]  John P. A. Ioannidis,et al.  The Power of Bias in Economics Research , 2017 .

[3]  Nathan D. Jones,et al.  Designing Large-Scale Multisite and Cluster-Randomized Studies of Professional Development , 2017 .

[4]  Annette N. Brown,et al.  The growth of impact evaluation for international development: how much have we learned? , 2016 .

[5]  Douglas L. Miller,et al.  A Practitioner’s Guide to Cluster-Robust Inference , 2015, The Journal of Human Resources.

[6]  J. Ioannidis,et al.  What's to Know About the Credibility of Empirical Economics? , 2013 .

[7]  A. Donner,et al.  Covariate-adjusted confidence interval for the intraclass correlation coefficient. , 2013, Contemporary clinical trials.

[8]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[9]  H. Bloom,et al.  New Empirical Evidence for the Design of Group Randomized Trials in Education , 2010 .

[10]  Joshua D. Angrist,et al.  The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con Out of Econometrics , 2010, SSRN Electronic Journal.

[11]  L. Hedges,et al.  Intraclass Correlation Values for Planning Group-Randomized Trials in Education , 2007 .

[12]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[13]  J. Grimshaw,et al.  Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported , 2004, BMC medical research methodology.

[14]  John P A Ioannidis,et al.  Evaluation of cluster randomized controlled trials in sub-Saharan Africa. , 2003, American journal of epidemiology.

[15]  S. Chinn,et al.  A note on the use of the variance inflation factor for determining sample size in cluster randomized trials , 2002 .

[16]  S. Chinn,et al.  Components of variance and intraclass correlations for the design of community-based surveys and intervention studies: data from the Health Survey for England 1994. , 1999, American journal of epidemiology.

[17]  Helen J. Craig,et al.  Schools Count: World Bank Project Designs and the Quality of Primary Education in Sub-Saharan Africa. Africa Technical Development Series. Technical Paper Number 303. , 1995 .

[18]  Dean Karlan,et al.  Six Randomized Evaluations of Microcredit: Introduction and Further Steps † , 2015 .