Outcomes and statistical power in adult critical care randomized trials.

RATIONALE Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically ill patients commonly fail to detect treatment benefits. OBJECTIVES Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials. METHODS One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied. MEASUREMENT AND MAIN RESULTS Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%. CONCLUSIONS ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficient patient accrual, and incorrect predictions of baseline event rates.

[1]  D. Moher,et al.  CONSORT for Reporting Randomized Controlled Trials in Journal and Conference Abstracts: Explanation and Elaboration , 2008, PLoS medicine.

[2]  S. Halpern.  Financial Incentives for Research Participation: Empirical Questions, Available Answers and the Burden of Further Proof , 2011, The American journal of the medical sciences.

[3]  D. Scales.  Research to inform the consent-to-research process , 2013, Intensive Care Medicine.

[4]  Hester F. Lingsma,et al.  Underpowered trials in critical care medicine: how to deal with them? , 2010, Critical care.

[5]  D. Dreyfuss.  Beyond randomized, controlled trials , 2004, Current opinion in critical care.

[6]  D. Needham,et al.  Beyond mortality: future clinical research in acute lung injury. , 2010, American journal of respiratory and critical care medicine.

[7]  A. Loundou,et al.  Neuromuscular blockers in early acute respiratory distress syndrome. , 2010, The New England journal of medicine.

[8]  C. Naylor,et al.  Can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials? , 1994, Journal of clinical epidemiology.

[9]  M. Griffiths,et al.  A stepwise approach to justify phase III randomized clinical trials and enhance the likelihood of a positive result , 2010, Critical care medicine.

[10]  J. Tenhunen,et al.  Hydroxyethyl starch 130/0.42 versus Ringer's acetate in severe sepsis. , 2012, The New England journal of medicine.

[11]  S. Aberegg,et al.  Delta inflation: a bias in the design of randomized controlled trials in critical care medicine , 2010, Critical care.

[12]  N. Latronico,et al.  Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010 , 2013, Intensive Care Medicine.

[13]  Comment: The Place of Death in the Quality of Life. , 2006, math/0612786.

[14]  G. Martin,et al.  Critical care trial design and interpretation: A primer , 2010, Critical care medicine.

[15]  Ewout W Steyerberg,et al.  Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. , 2004, Journal of clinical epidemiology.

[16]  Douglas G Altman,et al.  Systematic reviews in health care: Assessing the quality of controlled clinical trials. , 2001, BMJ.

[17]  J. Vincent,et al.  Improving clinical trials in the critically ill , 2010, Critical care medicine.

[18]  G. Van den Berghe,et al.  Intensive insulin therapy in the medical ICU. , 2006, The New England journal of medicine.

[19]  A. Garland,et al.  Physicians' influence over decisions to forego life support. , 2007, Journal of palliative medicine.

[20]  G. Guyatt,et al.  Integrating mortality and morbidity outcomes: using quality-adjusted life years in critical care trials. , 2013, American journal of respiratory and critical care medicine.

[21]  M R Petersen,et al.  Approaches for estimating prevalence ratios , 2008, Occupational and Environmental Medicine.

[22]  R. Bellomo,et al.  Hydroxyethyl starch or saline for fluid resuscitation in intensive care. , 2012, The New England journal of medicine.

[23]  J. Bakker,et al.  Early lactate-guided therapy in intensive care unit patients: a multicenter, open-label, randomized controlled trial. , 2010, American journal of respiratory and critical care medicine.

[24]  J. Marini.  Limitations of clinical trials in acute lung injury and acute respiratory distress syndrome , 2006, Current opinion in critical care.

[25]  P. Harris,et al.  Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support , 2009, J. Biomed. Informatics.

[26]  G. Rubenfeld,et al.  Estimating sample size in critical care clinical trials. , 2005, Journal of critical care.

[27]  S. Jaber,et al.  Prone positioning in severe acute respiratory distress syndrome. , 2013, The New England journal of medicine.

[28]  D. Annane.  Improving clinical trials in the critically ill: Unique challenge—Sepsis , 2009, Critical care medicine.

[29]  The randomized controlled trial needs critical care. , 2008, Critical care medicine.

[30]  Hester F. Lingsma,et al.  Baseline characteristics and statistical power in randomized controlled trials: Selection, prognostic targeting, or covariate adjustment?* , 2009, Critical care medicine.

[31]  R. Bellomo,et al.  End points for phase II trials in intensive care: recommendations from the Australian and New Zealand Clinical Trials Group consensus panel meeting. , 2012, Critical care and resuscitation : journal of the Australasian Academy of Critical Care Medicine.

[32]  J. Chiche,et al.  Testing protocols in the intensive care unit: complex trials of complex interventions for complex patients. , 2008, JAMA.

[33]  Meinhard Kieser,et al.  Sample Size Recalculation in Internal Pilot Study Designs: A Review , 2006, Biometrical journal. Biometrische Zeitschrift.

[34]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.

[35]  Hester F. Lingsma,et al.  A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT Project , 2010, Clinical trials.

[36]  J. Vincent,et al.  Multicenter, randomized, controlled trials evaluating mortality in intensive care: Doomed to fail? , 2008, Critical care medicine.

[37]  T. van der Poll,et al.  Severe sepsis and septic shock. , 2013, The New England journal of medicine.

[38]  D. Sackett.  Evidence-Based Medicine: How to Practice and Teach EBM , 2018 .

[39]  Andrew Rhodes,et al.  Drotrecogin alfa (activated) in adults with septic shock. , 2012, The New England journal of medicine.

[40]  David M Kent,et al.  Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. , 2007, JAMA.

[41]  J. Marshall,et al.  How outcomes are defined in clinical trials of mechanically ventilated adults and children. , 2014, American journal of respiratory and critical care medicine.

[42]  J. Vincent.  We should abandon randomized controlled trials in the intensive care unit , 2010, Critical care medicine.

[43]  Kert Viele,et al.  An Adaptive, Phase II, Dose-Finding Clinical Trial Design to Evaluate L-Carnitine in the Treatment of Septic Shock Based on Efficacy and Predictive Probability of Subsequent Phase III Success , 2013, Critical care medicine.

[44]  D. Schoenfeld,et al.  Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. , 2000, The New England journal of medicine.

[45]  D. Angus,et al.  The clinical research enterprise in critical care: What’s right, what’s wrong, and what’s ahead? , 2009, Critical care medicine.

[46]  Renda Soylemez Wiener,et al.  Two Decades of Mortality Trends Among Patients With Severe Sepsis: A Comparative Meta-Analysis* , 2014, Critical care medicine.

[47]  J. Karlawish,et al.  The continuing unethical conduct of underpowered clinical trials. , 2002, JAMA.

[48]  G. Rubenfeld,et al.  When is a negative phase II trial truly negative? , 2008, American journal of respiratory and critical care medicine.

[49]  G. Bernard,et al.  Efficacy and safety of a paired sedation and ventilator weaning protocol for mechanically ventilated patients in intensive care (Awakening and Breathing Controlled trial): a randomised controlled trial , 2008, The Lancet.

[50]  R. Holloway,et al.  Mortality as a measure of quality: implications for palliative and end-of-life care. , 2007, JAMA.

[51]  D. Kent,et al.  Competing risk and heterogeneity of treatment effect in clinical trials , 2008, Trials.

[52]  D. Cook,et al.  The ethical conduct of clinical research involving critically ill patients in the United States and Canada: principles and recommendations. , 2004, American journal of respiratory and critical care medicine.