The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies

BackgroundAdjustment for prognostic covariates can lead to increased power in the analysis of randomized trials. However, adjusted analyses are not often performed in practice.MethodsWe used simulation to examine the impact of covariate adjustment on 12 outcomes from 8 studies across a range of therapeutic areas. We assessed (1) how large an increase in power can be expected in practice; and (2) the impact of adjustment for covariates that are not prognostic.ResultsAdjustment for known prognostic covariates led to large increases in power for most outcomes. When power was set to 80% based on an unadjusted analysis, covariate adjustment led to a median increase in power to 92.6% across the 12 outcomes (range 80.6 to 99.4%). Power was increased to over 85% for 8 of 12 outcomes, and to over 95% for 5 of 12 outcomes. Conversely, the largest decrease in power from adjustment for covariates that were not prognostic was from 80% to 78.5%.ConclusionsAdjustment for known prognostic covariates can lead to substantial increases in power, and should be routinely incorporated into the analysis of randomized trials. The potential benefits of adjusting for a small number of possibly prognostic covariates in trials with moderate or large sample sizes far outweigh the risks of doing so, and so should also be considered.

[1]  Hester F. Lingsma,et al.  A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT Project , 2010, Clinical trials.

[2]  T. Pruett.  Intrapleural Use of Tissue Plasminogen Activator and DNase in Pleural Infection , 2012 .

[3]  Andrea Manca,et al.  A substantial and confusing variation exists in handling of baseline covariates in randomized controlled trials: a review of trials published in leading medical journals. , 2010, Journal of clinical epidemiology.

[4]  B. Kahan,et al.  Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis , 2012, BMJ : British Medical Journal.

[5]  Malcolm Kohler,et al.  Continuous positive airway pressure improves sleepiness but not calculated vascular risk in patients with minimally symptomatic obstructive sleep apnoea: the MOSAIC randomised controlled trial , 2012, Thorax.

[6]  Julius Sim,et al.  Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study , 2014, BMC Medical Research Methodology.

[7]  A. Mcgregor,et al.  ISSLS Prize Winner: Function After Spinal Treatment, Exercise, and Rehabilitation (FASTER) A Factorial Randomized Trial to Determine Whether the Functional Outcome of Spinal Surgery Can Be Improved , 2011, Spine.

[8]  D. Altman,et al.  Beneficial effect of azathioprine and prediction of prognosis in primary biliary cirrhosis. Final results of an international trial. , 1985, Gastroenterology.

[9]  S. Travis,et al.  Prevalence, management, and outcomes of patients with coagulopathy after acute nonvariceal upper gastrointestinal bleeding in the United Kingdom , 2013, Transfusion.

[10]  R. Kay.  Statistical Principles for Clinical Trials , 1998, The Journal of international medical research.

[11]  S. Lipsitz,et al.  Does Clustering Affect the Usual Test Statistics of no Treatment Effect in a Randomized Clinical Trial , 1998 .

[12]  Stephen Senn,et al.  Statistical Issues in Drug Development , 1997 .

[13]  C. Doré,et al.  National audit of the use of surgery and radiological embolization after failed endoscopic haemostasis for non‐variceal upper gastrointestinal bleeding , 2012, The British journal of surgery.

[14]  S. Assmann,et al.  Subgroup analysis and other (mis)uses of baseline data in clinical trials , 2000, The Lancet.

[15]  J. Hanley,et al.  The effect of omitted covariates on confidence interval and study power in binary outcome analysis: a simulation study. , 2007, Contemporary clinical trials.

[16]  G. Breukelen.  ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies , 2006 .

[17]  David Moher,et al.  SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials , 2013, BMJ.

[18]  Marion K Campbell,et al.  The method of minimization for allocation to clinical trials. a review. , 2002, Controlled clinical trials.

[19]  R. Davies,et al.  Effect of an indwelling pleural catheter vs chest tube and talc pleurodesis for relieving dyspnea in patients with malignant pleural effusion: the TIME2 randomized controlled trial. , 2012, JAMA.

[20]  C. Doré,et al.  Granulocyte-macrophage colony stimulating factor administered as prophylaxis for reduction of sepsis in extremely preterm, small for gestational age neonates (the PROGRAMS trial): a single-blind, multicentre, randomised controlled trial , 2009, The Lancet.

[21]  D. Stoyan.  Stereology and stochastic geometry , 1990 .

[22]  A. Agresti,et al.  Strategies for comparing treatments on a binary response with multi-centre data. , 2000, Statistics in medicine.

[23]  Nick C Fox,et al.  Reduced sample sizes for atrophy outcomes in Alzheimer's disease trials: baseline adjustment , 2010, Neurobiology of Aging.

[24]  Nazmus Saquib,et al.  Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study , 2013, BMJ.

[25]  Ian R White,et al.  Adjusting for partially missing baseline measurements in randomized trials , 2005, Statistics in medicine.

[26]  Ewout W Steyerberg,et al.  Randomized controlled trials with time-to-event outcomes: how much does prespecified covariate adjustment increase power? , 2006, Annals of epidemiology.

[27]  Gerard J P Van Breukelen,et al.  ANCOVA versus change from baseline: more power in randomized studies, more bias in nonrandomized studies [corrected]. , 2006, Journal of clinical epidemiology.

[28]  B. Kahan,et al.  Improper analysis of trials randomised using stratified blocks or minimisation , 2012, Statistics in medicine.

[29]  B. Kahan,et al.  Analysis of multicentre trials with continuous outcomes: when and how should we account for centre effects? , 2013, Statistics in medicine.

[30]  Van Breukelen Gj.  ANCOVA versus change from baseline: more power in randomized studies, more bias in nonrandomized studies [corrected]. , 2006 .

[31]  J. Matthews,et al.  Randomization in Clinical Trials: Theory and Practice; , 2003 .

[32]  B. Kahan,et al.  Adjusting for multiple prognostic factors in the analysis of randomised trials , 2013, BMC Medical Research Methodology.

[33]  J. Habbema,et al.  Subgroup Analysis and Covariate Adjustment in Randomized Clinical Trials of Traumatic Brain Injury: A Systematic Review , 2005, Neurosurgery.

[34]  S. Pocock,et al.  Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practiceand problems , 2002, Statistics in medicine.

[35]  D R Taves,et al.  Minimization: A new method of assigning patients to treatment and control groups , 1974, Clinical pharmacology and therapeutics.

[36]  E. Steyerberg,et al.  Adjustment for strong predictors of outcome in traumatic brain injury trials: 25% reduction in sample size requirements in the IMPACT study. , 2006, Journal of neurotrauma.

[37]  ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials. International Conference on Harmonisation E9 Expert Working Group. , 1999, Statistics in medicine.

[38]  Anika Ashok,et al.  ICH Harmonised Tripartite Guideline , 2009 .

[39]  D. Altman,et al.  Reporting on covariate adjustment in randomised controlled trials before and after revision of the 2001 CONSORT statement: a literature review , 2010, Trials.

[40]  W W Hauck,et al.  Should we adjust for covariates in nonlinear regression analyses of randomized trials? , 1998, Controlled clinical trials.

[41]  A. Holbrook,et al.  Comparing methods to estimate treatment effects on a continuous outcome in multicentre randomized controlled trials: A simulation study , 2011, BMC medical research methodology.

[42]  S. Pocock,et al.  Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. , 1975, Biometrics.

[43]  S. Travis,et al.  Mortality From Acute Upper Gastrointestinal Bleeding in the United Kingdom: Does It Display a “Weekend Effect”? , 2011, The American Journal of Gastroenterology.

[44]  P. Fayers,et al.  Interferon-alpha and survival in metastatic renal carcinoma: early results of a randomised controlled trial , 1999 .

[45]  Ewout W Steyerberg,et al.  Covariate adjustment increased power in randomized controlled trials: an example in traumatic brain injury. , 2012, Journal of clinical epidemiology.

[46]  N. Jewell,et al.  Some surprising results about covariate adjustment in logistic regression models , 1991 .

[47]  Medical Research Council Renal Cancer Collaborators.  Interferon-α and survival in metastatic renal carcinoma: early results of a randomised controlled trial , 1999, The Lancet.

[48]  C. Doré,et al.  Outcomes following acute nonvariceal upper gastrointestinal bleeding in relation to time to endoscopy: results from a nationwide study , 2012, Endoscopy.

[49]  Ewout W Steyerberg,et al.  Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. , 2004, Journal of clinical epidemiology.

[50]  Gareth Griffiths,et al.  Interferon-alpha and survival in metastatic renal carcinoma: early results of a randomised controlled trial. Medical Research Council Renal Cancer Collaborators. , 1999, Lancet.

[51]  G M Raab,et al.  How to select covariates to include in the analysis of a clinical trial. , 2000, Controlled clinical trials.

[52]  B. Kahan,et al.  Assessing potential sources of clustering in individually randomised trials , 2013, BMC Medical Research Methodology.