Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health.

OBJECTIVES To illustrate how multiple hypotheses testing can produce associations with no clinical plausibility. STUDY DESIGN AND SETTING We conducted a study of all 10,674,945 residents of Ontario aged between 18 and 100 years in 2000. Residents were randomly assigned to equally sized derivation and validation cohorts and classified according to their astrological sign. Using the derivation cohort, we searched through 223 of the most common diagnoses for hospitalization until we identified two for which subjects born under one astrological sign had a significantly higher probability of hospitalization compared to subjects born under the remaining signs combined (P<0.05). RESULTS We tested these 24 associations in the independent validation cohort. Residents born under Leo had a higher probability of gastrointestinal hemorrhage (P=0.0447), while Sagittarians had a higher probability of humerus fracture (P=0.0123) compared to all other signs combined. After adjusting the significance level to account for multiple comparisons, none of the identified associations remained significant in either the derivation or validation cohort. CONCLUSIONS Our analyses illustrate how the testing of multiple, non-prespecified hypotheses increases the likelihood of detecting implausible associations. Our findings have important implications for the analysis and interpretation of clinical studies.

[1]  G H Guyatt,et al.  A Consumer's Guide to Subgroup Analyses , 1992, Annals of Internal Medicine.

[2]  D. Phillips,et al.  Psychology and survival , 1993, The Lancet.

[3]  Nick Freemantle,et al.  Interpreting the results of secondary end points and subgroup analyses in clinical trials: should we lock the crazy aunt in the attic? , 2001, BMJ : British Medical Journal.

[4]  Psychology and survival , 1994, The Lancet.

[5]  Predicting incidence of some critical events by sun signs—the Pisces study , 2003 .

[6]  J. Cleland,et al.  Clinical trials update: OPTIME‐CHF, PRAISE‐2, ALL‐HAT , 2000, European journal of heart failure.

[7]  G GordilloPaniagua,et al.  The International Committee of Medical Journal Editors , 1981 .

[8]  B. Everitt The Cambridge Dictionary of Statistics , 1998 .

[9]  L. Bolognese,et al.  RANDOMISED TRIAL OF INTRAVENOUS STREPTOKINASE, ORAL ASPIRIN, BOTH, OR NEITHER AMONG 17 187 CASES OF SUSPECTED ACUTE MYOCARDIAL INFARCTION: ISIS-2 , 1988, The Lancet.

[10]  E. Veys,et al.  HL-A AND INFECTIVE SACROILEITIS , 1974 .

[11]  M F Huque,et al.  Some comments on frequently used multiple endpoint adjustment methods in clinical trials. , 1997, Statistics in medicine.

[12]  S. Lange,et al.  Adjusting for multiple testing--when and how? , 2001, Journal of clinical epidemiology.

[13]  P. Rothwell Subgroup analysis in randomised controlled trials: importance, indications, and interpretation , 2005, The Lancet.

[14]  J. Wittes,et al.  Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. , 1991, JAMA.

[15]  Bertram Pitt,et al.  Randomised trial of losartan versus captopril in patients over 65 with heart failure (Evaluation of Losartan in the Elderly Study, ELITE) , 1997, The Lancet.

[16]  L. J. Anthony,et al.  The Cambridge Dictionary of Statistics (2nd ed.) , 2003 .

[17]  P. Sleight Debate: Subgroup analyses in clinical trials: fun to look at - but don't believe them! , 2000, Current controlled trials in cardiovascular medicine.

[18]  John Goodier,et al.  The Cambridge Dictionary of Statistics (3rd edition) , 2007 .

[19]  Eter,et al.  EFFECT OF AMLODIPINE ON MORBIDITY AND MORTALITY IN SEVERE CHRONIC HEART FAILURE , 2000 .

[20]  K. Lee,et al.  Perspectives on large-scale cardiovascular clinical trials for the new millennium. The Virtual Coordinating Center for Global Collaborative Cardiovascular Research (VIGOUR) Group. , 1997, Circulation.

[21]  Bertram Pitt,et al.  Effect of losartan compared with captopril on mortality in patients with symptomatic heart failure: randomised trial—the Losartan Heart Failure Survival Study ELITE II , 2000, The Lancet.

[22]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2005, Circulation.