Power Analysis for Two-Level Multisite Randomized Cost-Effectiveness Trials

Cost-effectiveness analysis is a widely used educational evaluation tool. The randomized controlled trials that aim to evaluate the cost-effectiveness of the treatment are commonly referred to as randomized cost-effectiveness trials (RCETs). This study provides methods of power analysis for two-level multisite RCETs. Power computations take account of sample sizes, the effect size, covariates effects, nesting effects for both cost and effectiveness measures, the ratio of the total variance of the cost measure to the total variance of effectiveness measure, and correlations between cost and effectiveness measures at each level. Illustrative examples that show how power is influenced by the sample sizes, nesting effects, covariate effects, and correlations between cost and effectiveness measures are presented. We also demonstrate how the calculations can be applied in the design phase of two-level multisite RCETs using the software PowerUp!-CEA (Version 1.0).

[1]  Md Abu Manju,et al.  Optimal and maximin sample sizes for multicentre cost-effectiveness trials , 2015, Statistical methods in medical research.

[2]  R. Chisin.  Cost-Effectiveness Analysis , 2009, Journal of Nuclear Medicine.

[3]  Patrick J. Wolf,et al.  Is the Juice Worth the Squeeze? A Benefit/Cost Analysis of the District of Columbia Opportunity Scholarship Program , 2013, Education Finance and Policy.

[4]  Spyros Konstantopoulos,et al.  The Power of the Test for Treatment Effects in Three-Level Cluster Randomized Designs , 2008 .

[5]  Rebecca Maynard,et al.  PowerUp!: A Tool for Calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes for Experimental and Quasi-Experimental Design Studies , 2013 .

[6]  Gema Zamarro,et al.  Dual-Language Immersion Education at Scale: An Analysis of Program Costs, Mechanisms, and Moderators , 2018, Educational Evaluation and Policy Analysis.

[7]  Peter Muennig,et al.  Cost-effectiveness analyses in health : a practical approach , 2008 .

[8]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[9]  Henry M. Levin,et al.  Waiting for Godot: Cost‐Effectiveness Analysis in Education , 2001 .

[10]  S. Raudenbush,et al.  Statistical power and optimal design for multisite randomized trials. , 2000, Psychological methods.

[11]  Gerard J. P. Van Breukelen,et al.  SamP2CeT: an interactive computer program for sample size and power calculation for two-level cost-effectiveness trials , 2018, Comput. Stat..

[12]  Yilin Pan,et al.  Cost-Effectiveness Analysis of Early Reading Programs: A Demonstration With Recommendations for Future Research , 2016 .

[13]  Spyros Konstantopoulos,et al.  The Power of the Test for Treatment Effects in Three-Level Block Randomized Designs , 2008 .

[14]  S. Konstantopoulos.  Power Analysis in Two-Level Unbalanced Designs , 2010 .

[15]  Douglas N. Harris,et al.  Toward Policy-Relevant Benchmarks for Interpreting Effect Sizes: Combining Effects With Costs , 2009 .

[16]  John Mullahy,et al.  Net Health Benefits: A New Framework for the Analysis of Uncertainty in Cost-Effectiveness Analysis , 1998 .

[17]  Robert F. Boruch,et al.  Sensitivity, bias, and theory in impact evaluations. , 1977 .

[18]  N. Dong,et al.  Bound Constrained Optimization of Sample Sizes Subject to Monetary Restrictions in Planning Multilevel Randomized Trials and Regression Discontinuity Studies , 2019, The Journal of Experimental Education.

[19]  Larry V. Hedges,et al.  Statistical Power Analysis in Education Research , 2010 .

[20]  Benjamin Kelcey,et al.  Optimal Design of Cluster- and Multisite-Randomized Studies Using Fallible Outcome Measures , 2019, Evaluation review.

[21]  A R Willan,et al.  Confidence intervals for cost-effectiveness ratios: an application of Fieller's theorem. , 1996, Health economics.

[22]  Robin Jacob,et al.  Leveraging Volunteers: An Experimental Evaluation of a Tutoring Program for Struggling Readers , 2016 .

[23]  Jessaca Spybrook,et al.  Sample Size Planning for Cluster-Randomized Interventions Probing Multilevel Mediation , 2018, Prevention Science.

[24]  C. Belfield,et al.  Guiding the Development and Use of Cost-Effectiveness Analysis in Education , 2015 .

[25]  Jessaca Spybrook,et al.  Optimal Design for Longitudinal and Multilevel Research: Documentation for the "Optimal Design" Software , 2006 .

[26]  S. Raudenbush.  Statistical analysis and optimal design for cluster randomized trials , 1997 .

[27]  Rajeev Kumar,et al.  Sample size calculation , 2012, Indian journal of ophthalmology.

[28]  Jessaca K. Spybrook,et al.  Power Analyses for Moderator Effects in Three-Level Cluster Randomized Trials , 2018 .

[29]  E. Wagner,et al.  A multisite randomized trial of the effects of physician education and organizational change in chronic-asthma care: health outcomes of the Pediatric Asthma Care Patient Outcomes Research Team II Study. , 2004, Archives of pediatrics & adolescent medicine.

[30]  Larry V. Hedges,et al.  Intraclass Correlations for Planning Group Randomized Experiments in Rural Education , 2007 .

[31]  S. Konstantopoulos.  The Impact of Covariates on Statistical Power in Cluster Randomized Designs: Which Level Matters More? , 2012, Multivariate behavioral research.

[32]  Clive R. Belfield,et al.  Using Resource and Cost Considerations to Support Educational Evaluation: Six Domains , 2018, Educational Researcher.

[33]  Kevin B Weiss,et al.  A multisite randomized trial of the effects of physician education and organizational change in chronic asthma care: cost-effectiveness analysis of the Pediatric Asthma Care Patient Outcomes Research Team II (PAC-PORT II). , 2005, Archives of pediatrics & adolescent medicine.

[34]  H. Bloom,et al.  Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational Interventions , 2007 .

[35]  E. Elbasha.  Statistical Analysis of Cost-Effectiveness Data , 2008 .

[36]  S. Konstantopoulos,et al.  Power Analysis for Models of Change in Cluster Randomized Designs , 2017, Educational and psychological measurement.

[37]  Diana B. Petitti,et al.  Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis: Methods for Quantitative Synthesis in Medicine , 1994 .

[38]  B. Shaywitz,et al.  Evaluating Willingness to Pay as a Measure of the Impact of Dyslexia in Adults , 2017, Journal of Benefit-Cost Analysis.

[39]  Md Abu Manju,et al.  Sample size calculation in cost‐effectiveness cluster randomized trials: optimal and maximin approaches , 2014, Statistics in medicine.