Re: Low-fat dietary pattern and cancer incidence in the Women's Health Initiative Dietary Modification Randomized Controlled Trial.

Making multiple claims from one data set requires some care with the statistical analysis; otherwise, scientists making the claims and their readers might easily be fooled by randomness. In the Woman’s Health Initiative Dietary Modification Trial, two primary questions related to a low-fat vs a normal diet and cancer risk were examined ( 1 – 2 ). These hypotheses were motivated by extensive epidemiology work and were expected to be true based on considerable evidence. In fact, neither of these claims was validated in the randomized trial. That these claims were not validated surprised some people, but the results should have been expected; Ioannidis ( 3 ) pointed out that claims from nonrandomized medical studies fail to replicate approximately 80% of the time. One reason ( 4 ) that claims coming from epidemiology studies fail to replicate is that epidemiologists routinely do not correct their statistical analysis for multiple testing ( 5 ). One of the simplest ways to correct for multiple testing is to use the Bonferroni correction, whereby the number of questions under consideration is counted and the unadjusted P values are multiplied by the number of questions. More sophisticated methods are available [( 6 ) among many others]. Of course, the data analysis strategy and the counting should be specifi ed before the data are examined. Counting the questions Prentice et al. ( 7 ) have under consideration is a bit elusive. There are two named primary questions, breast and colorectal cancers. A normal clinical trial strategy would be to multiply each by two to give adjusted P values; any possible effects on secondary endpoints would need to be verifi ed in a new study. In fact, fi ve types of cancer are under consideration: breast, colon, rectum, ovary, and endometrium, as are three time periods: the fi rst 4 years, the next 4.1 years, and total, giving a Bonferroni multiplier of 15. There are more than 60 P values reported in tables 1 – 5, and none are adjusted for multiple comparisons. It is possible that many additional P values were computed but are not reported. The three smallest P values are .03, .05, and .05. Any reasonable adjustment renders these P values non – statistically signifi cant; seeing a P value as small as .03 is quite likely by chance alone. In clinical trials, results must be statistically signifi cant in two studies to make a claim, and the usual convention of science is that anyone wishing to assert a claim should have strong supporting evidence. Although technically correct, the summary statement “A low-fat dietary pattern may reduce ... ” could easily be misinterpreted. More accurate would be the statement, “Epidemiology studies suggested that fi ve types of cancer might be reduced by adhering to a low-fat diet. A well-conducted, large, randomized trial gives no statistical support to any of these claims.”