Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis.

CONTEXT Theory and simulation suggest that randomized controlled trials (RCTs) stopped early for benefit (truncated RCTs) systematically overestimate treatment effects for the outcome that precipitated early stopping. OBJECTIVE To compare the treatment effect from truncated RCTs with that from meta-analyses of RCTs addressing the same question but not stopped early (nontruncated RCTs) and to explore factors associated with overestimates of effect. DATA SOURCES Search of MEDLINE, EMBASE, Current Contents, and full-text journal content databases to identify truncated RCTs up to January 2007; search of MEDLINE, Cochrane Database of Systematic Reviews, and Database of Abstracts of Reviews of Effects to identify systematic reviews from which individual RCTs were extracted up to January 2008. STUDY SELECTION Selected studies were RCTs reported as having stopped early for benefit and matching nontruncated RCTs from systematic reviews. Independent reviewers with medical content expertise, working blinded to trial results, judged the eligibility of the nontruncated RCTs based on their similarity to the truncated RCTs. DATA EXTRACTION Reviewers with methodological expertise conducted data extraction independently. RESULTS The analysis included 91 truncated RCTs asking 63 different questions and 424 matching nontruncated RCTs. The pooled ratio of relative risks in truncated RCTs vs matching nontruncated RCTs was 0.71 (95% confidence interval, 0.65-0.77). This difference was independent of the presence of a statistical stopping rule and the methodological quality of the studies as assessed by allocation concealment and blinding. Large differences in treatment effect size between truncated and nontruncated RCTs (ratio of relative risks <0.75) occurred with truncated RCTs having fewer than 500 events. In 39 of the 63 questions (62%), the pooled effects of the nontruncated RCTs failed to demonstrate significant benefit. CONCLUSIONS Truncated RCTs were associated with greater effect sizes than RCTs not stopped early. This difference was independent of the presence of statistical stopping rules and was greatest in smaller studies.

[1]  B. Freidlin,et al.  Stopping clinical trials early for benefit: impact on estimation , 2009, Clinical trials.

[2]  G. Guyatt,et al.  Stopping randomized trials early for benefit: a protocol of the Study Of Trial Policy Of Interim Truncation-2 (STOPIT-2) , 2009, Trials.

[3]  K. Schulz,et al.  Multiplicity in randomised trials II: subgroup and interim analyses , 2005, The Lancet.

[4]  Sydes,et al.  A proposed charter for clinical trial data monitoring committees: helping them to do their job well , 2005, The Lancet.

[5]  Ron-Li Liaw,et al.  Department of , 1993 .

[6]  D. Clayton,et al.  Be skeptical about unexpected large apparent treatment effects: the case of an MRC AML12 randomization. , 2003, Controlled clinical trials.

[7]  B. Freidlin,et al.  Stopping or reporting early for positive results in randomized clinical trials: the National Cancer Institute Cooperative Group experience from 1990 to 2005. , 2009, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[8]  S. Goodman.  Stopping at Nothing? Some Dilemmas of Data Monitoring in Clinical Trials , 2007, Annals of Internal Medicine.

[9]  Stuart J Pocock,et al.  Current controversies in data monitoring for clinical trials , 2006, Clinical trials.

[10]  Gordon H Guyatt,et al.  Randomized trials stopped early for benefit: a systematic review. , 2005, JAMA.

[11]  D. Rennie,et al.  Stopping medical research to save money: a broken pact with researchers and patients. , 2003, JAMA.

[12]  S. Pocock.  When (not) to stop a clinical trial for benefit. , 2005, JAMA.

[13]  G. Norman,et al.  Interpretation of Changes in Health-related Quality of Life: The Remarkable Universality of Half a Standard Deviation , 2003, Medical care.

[14]  Interim monitoring of efficacy data is important and appropriate. , 2008, Journal of clinical epidemiology.

[15]  S. Goodman.  Stopping trials for efficacy: an almost unbiased view , 2009, Clinical trials.

[16]  G. Guyatt,et al.  Ethical Issues in Stopping Randomized Trials Early Because of Apparent Benefit , 2007, Annals of Internal Medicine.

[17]  S. Pocock,et al.  Trials stopped early: too good to be true? , 1999, The Lancet.

[18]  M D Hughes,et al.  Practical problems in interim analyses, with particular regard to estimation. , 1989, Controlled clinical trials.

[19]  Matthias Briel,et al.  Early stopping of randomized clinical trials for overt efficacy is problematic. , 2008, Journal of clinical epidemiology.

[20]  E. Eisenhauer,et al.  Stopping a trial early in oncology: for patients or for industry? , 2008, Annals of oncology : official journal of the European Society for Medical Oncology.

[21]  Sally Hopewell,et al.  Publication bias in clinical trials due to statistical significance or direction of trial results. , 2009, The Cochrane database of systematic reviews.

[22]  D. Sargent.  Early stopping for benefit in National Cancer Institute-sponsored randomized Phase III trials: the system is working. , 2009, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[23]  G. Guyatt,et al.  Systematic reviewers neglect bias that results from trials stopped early for benefit. , 2007, Journal of clinical epidemiology.