Sample size and power calculations for medical studies by simulation when closed form expressions are not available

This paper shows how Monte Carlo simulation can be used for sample size, power or precision calculations when planning medical research studies. Standard study designs can lead to the use of analysis methods for which power formulae do not exist. This may be because complex modelling techniques with optimal statistical properties are used but power formulae have not yet been derived or because analysis models are employed that divert from the population model due to lack of availability of more appropriate analysis tools. Our presentation concentrates on the conceptual steps involved in carrying out power or precision calculations by simulation. We demonstrate these steps in three examples concerned with (i) drop out in longitudinal studies, (ii) measurement error in observational studies and (iii) causal effect estimation in randomised controlled trials with non-compliance. We conclude that the Monte Carlo simulation approach is an important general tool in the methodological arsenal for assessing power and precision.

[1]  Xitao Fan,et al.  Assessing the Reliability of Beck Depression Inventory Scores: Reliability Generalization across Studies , 2000 .

[2]  A Donner,et al.  Developments in cluster randomized trials and Statistics in Medicine , 2007, Statistics in medicine.

[3]  Donald A. Berry,et al.  Relationship Between Bayesian and Frequentist Sample Size Determination , 2005 .

[4]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[5]  Patrick Royston,et al.  The cost of dichotomising continuous variables , 2006, BMJ : British Medical Journal.

[6]  Allan Donner,et al.  Sample size requirements for the design of reliability study: review and new results , 2004 .

[7]  J. Hardin,et al.  Generalized Linear Models and Extensions , 2001 .

[8]  Leona S. Aiken,et al.  Loss of Power in Logistic, Ordinal Logistic, and Probit Regression When an Outcome Variable Is Coarsely Categorized , 2006 .

[9]  C. Dowrick,et al.  Estimating psychological treatment effects from a randomised controlled trial with both non-compliance and loss to follow-up , 2003, British Journal of Psychiatry.

[10]  Simon G Thompson,et al.  Allowing for imprecision of the intracluster correlation coefficient in the design of cluster randomized trials , 2004, Statistics in medicine.

[11]  B. Muthén,et al.  How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power , 2002 .

[12]  Booil Jo,et al.  Statistical power in randomized intervention studies with noncompliance. , 2002, Psychological methods.

[13]  Roger A. Sugden,et al.  Multiple Imputation for Nonresponse in Surveys , 1988 .

[14]  M. A. Best.  Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[15]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[16]  A. H. Feiveson,et al.  Power by Simulation , 2002 .

[17]  P Bélisle,et al.  Bayesian and mixed Bayesian/likelihood criteria for sample size determination. , 1997, Statistics in medicine.

[18]  J. Karlawish,et al.  The continuing unethical conduct of underpowered clinical trials. , 2002, JAMA.

[19]  Stuart J Pocock,et al.  Categorisation of continuous risk factors in epidemiological publications: a survey of current practice , 2010, Epidemiologic perspectives & innovations : EP+I.

[20]  Henian Chen,et al.  Biased odds ratios from dichotomization of age , 2006, Statistics in medicine.

[21]  Donald B Rubin,et al.  Principal Stratification for Causal Inference With Extended Partial Compliance , 2008 .

[22]  R. J. Shephard,et al.  Comparison of adaptive pacing therapy, cognitive behaviour therapy, graded exercise therapy, and specialist medical care for chronic fatigue syndrome (PACE): a randomised trial , 2012 .

[23]  Howard S. Bloom,et al.  Accounting for No-Shows in Experimental Evaluation Designs , 1984 .

[24]  S Greenland,et al.  Avoiding power loss associated with categorization and ordinal scores in dose-response and trend analysis. , 1995, Epidemiology.

[25]  E N Atkinson,et al.  Projection from previous studies: a Bayesian and frequentist compromise. , 1987, Controlled clinical trials.

[26]  C. Adcock.  Sample size determination : a review , 1997 .

[27]  T. T. Have,et al.  An introduction to causal modeling in clinical trials , 2007, Clinical trials.

[28]  D. Katz.  The American Statistical Association , 2000 .

[29]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[30]  D. Ruppert,et al.  Measurement Error in Nonlinear Models , 1995 .

[31]  Maria L. Rizzo,et al.  Statistical Computing with R , 2007 .

[32]  Xihong Lin,et al.  A Comparison of Methods for Estimating the Causal Effect of a Treatment in Randomized Clinical Trials Subject to Noncompliance , 2009, Biometrics.

[33]  D. Rubin,et al.  Principal Stratification in Causal Inference , 2002, Biometrics.

[34]  D. Rubin.  Causal Inference Using Potential Outcomes , 2005 .

[35]  D. Richardson.  Power calculations for survival analyses via Monte Carlo estimation. , 2003, American journal of industrial medicine.

[36]  R. Little,et al.  Statistical Techniques for Analyzing Data from Prevention Trials: Treatment of No-Shows Using Rubin's Causal Model , 1998 .

[37]  MICHAEL S. Williams,et al.  Monte Carlo approaches for determining power and sample size in low-prevalence applications. , 2007, Preventive veterinary medicine.

[38]  Scott E Maxwell,et al.  Power in randomized group comparisons: the value of adding a single intermediate time point to a traditional pretest-posttest design. , 2002, Psychological methods.

[39]  S. Raudenbush,et al.  Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. , 2001, Psychological methods.

[40]  I. White.  Uses and limitations of randomization-based efficacy estimators , 2005, Statistical methods in medical research.

[41]  Alex H. S. Harris,et al.  Physical activity, exercise coping, and depression in a 10-year cohort study of depressed patients. , 2006, Journal of affective disorders.

[42]  D. Hedeker,et al.  Sample Size Estimation for Longitudinal Designs with Attrition: Comparing Time-Related Contrasts Between Two Groups , 1999 .

[43]  Bengt Muthén,et al.  Modeling of Intervention Effects With Noncompliance: A Latent Variable Approach for Randomized Trials , 2001 .

[44]  Marvin Zelen,et al.  Clinical Trials and Sample Size Considerations: Another Perspective , 2000 .

[45]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[46]  David L Streiner,et al.  Breaking up is Hard to Do: The Heartbreak of Dichotomizing Continuous Data , 2002, Canadian journal of psychiatry. Revue canadienne de psychiatrie.

[47]  Subhash Aryal,et al.  Sample Size Determination for Hierarchical Longitudinal Designs with Differential Attrition Rates , 2007, Biometrics.

[48]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[49]  Victor M. Gastañaga,et al.  Power calculations for generalized linear models in observational longitudinal studies: A simulation approach in SAS , 2006, Comput. Methods Programs Biomed..

[50]  D. Spiegelman,et al.  Power and sample size calculations for longitudinal studies estimating a main effect of a time-varying exposure , 2011, Statistical methods in medical research.

[51]  L. Corrado.  Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models , 2005 .

[52]  William J. Browne,et al.  A guide to sample size calculations for random effect models via simulation and the MLPowSim Software Package , 2009 .

[53]  Jan Palczewski,et al.  Monte Carlo Simulation , 2008, Encyclopedia of GIS.

[54]  X M Tu,et al.  Power analyses for longitudinal study designs with missing data , 2007, Statistics in medicine.

[55]  Booil Jo,et al.  Estimation of Intervention Effects with Noncompliance: Alternative Model Specifications , 2002 .

[56]  R. Serlin,et al.  Testing for robustness in Monte Carlo studies. , 2000, Psychological methods.

[57]  Hamid Pezeshk,et al.  Bayesian techniques for sample size determination in clinical trials: a short review , 2003, Statistical methods in medical research.

[58]  Jacob Cohen.  The Cost of Dichotomization , 1983 .

[59]  Ronald W. Shonkwiler,et al.  Explorations in Monte Carlo Methods , 2009 .

[60]  Shawna V. Hudson,et al.  Intraclass correlation estimates for cancer screening outcomes: estimates and applications in the design of group-randomized cancer screening studies. , 2010, Journal of the National Cancer Institute. Monographs.

[61]  Booil Jo,et al.  Statistical power in randomized intervention studies with noncompliance. , 2002 .

[62]  S. Raudenbush.  Statistical analysis and optimal design for cluster randomized trials , 1997 .

[63]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[64]  Fei Wang,et al.  A simulation-based approach to Bayesian sample size determination for performance under a given model and for separating models , 2002 .

[65]  Z Feng,et al.  Correlated binomial variates: properties of estimator of intraclass correlation and its effect on sample size calculation. , 1992, Statistics in medicine.