Estimating risk from underpowered, but statistically significant, studies: was APPROVe on TARGET?

What is known and objective:  The importance of statistical power is widely recognized from a pre‐trial perspective, and when interpreting results that are not statistically significant. It is less well recognized that poor power can lead to inflated estimates of the effect size when statistically significant results are observed. We use trial simulations to quantify this bias, which we term ‘significant‐result bias’.

[1]  Robert Riddell,et al.  Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. , 2005, The New England journal of medicine.

[2]  C Poole,et al.  Low P-Values or Narrow Confidence Intervals: Which Are More Durable? , 2001, Epidemiology.

[3]  C R Weinberg.  It's time to rehabilitate the P-value. , 2001, Epidemiology.

[4]  J. Emberson,et al.  Do selective cyclo-oxygenase-2 inhibitors and traditional non-steroidal anti-inflammatory drugs increase the risk of atherothrombosis? Meta-analysis of randomised trials , 2006, BMJ : British Medical Journal.

[5]  T. Schnitzer,et al.  Comparison of lumiracoxib with naproxen and ibuprofen in the Therapeutic Arthritis Research and Gastrointestinal Event Trial (TARGET), cardiovascular outcomes: randomised controlled trial , 2004, The Lancet.

[6]  C. Lines,et al.  Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. , 2005, The New England journal of medicine.

[7]  George Davey Smith,et al.  Misleading meta-analysis , 1995, BMJ.

[8]  D. DeMets,et al.  The need for large-scale randomized evidence without undue emphasis on small trials, meta-analyses, or subgroup analyses. , 2009, JAMA.

[9]  R. Sperling,et al.  Selective COX-2 inhibition and cardiovascular effects: a review of the rofecoxib development program. , 2003, American heart journal.

[10]  T. Peters,et al.  Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. , 2001, Health technology assessment.

[11]  C E Land,et al.  Estimating cancer risks from low doses of ionizing radiation. , 1980, Science.

[12]  A. Brody,et al.  Nonselective Nonsteroidal Antiinflammatory Drugs and Cardiovascular Risk: Are They Safe? , 2007, The Annals of pharmacotherapy.

[13]  S. Goodman,et al.  Of P-values and Bayes: a modest proposal. , 2001, Epidemiology.

[14]  Thomas A. Louis,et al.  An Assessment of Publication Bias Using a Sample of Published Clinical Trials , 1989 .

[15]  D. DeMets,et al.  Doing more good than harm: need for a cease fire. , 2009, The American journal of medicine.

[16]  R. Rettig,et al.  Food and Drug Administration Advisory Committees , 1992 .

[17]  R Peto,et al.  Large-scale randomized evidence: large, simple trials and overviews of trials. , 1993, Annals of the New York Academy of Sciences.