The importance of statistical power analysis: an example from Animal Behaviour

Statistical significance and biological significance are not the same thing. For example, given a large enough sample size, any statistical hypothesis test is likely to be statistically significant, almost regardless of the biological importance of the results. Conversely, when the sample size is small, biologically interesting phenomena may be missed because statistical tests are unlikely to yield statistically significant results. Statistical and biological significance can be linked through the use of statistical power analysis. While power analysis is gaining popularity in many branches of biology (reviewed by Fairweather 1991; Taylor & Gerodette 1993; Searcy-Bernal 1994), it has been largely ignored in others, including animal behaviour, despite attempts to draw it to general attention (Greenwood 1993). The purpose of this article is to reinforce Greenwood’s advice by clearly demonstrating the relationship between sample size, biological significance and statistical power and by providing key references to introductory papers, texts and computer software. We use an example from Animal Behaviour to illustrate the importance of power analysis and the consequences of ignoring power. Our example is taken from a recent aquarium study on the willingness of juvenile rainbow trout, Oncorhynchus mykiss, to forage under the risk of predation (Johnsson 1993). As a part of the study, the investigator tested the null hypothesis that large and small juvenile trout do not differ in their susceptibility to predation. To test this hypothesis, eight replicate groups of six large and six small juveniles were exposed one by one to a standardized encounter with a predatory adult trout. On average 19&4.9% (X&) of the large fish and 45&7.0% of the small fish were killed by the predator. The difference between the two size classes was not statistically significant using a Wilcoxon signed-ranks test and a significance criterion of a=0.05 (T=29, N=8, P=0.15). Does this mean that the null hypothesis of no difference should be accepted? Not necessarily: another possibility is that there exists a biologically significant difference in susceptibility to predation in the population, but that the test was not sensitive enough to detect it. Statistical power analysis allows the evaluation of these two alternatives. The statistical power of a test is the probability of getting a statistically significant result, given that the null hypothesis is false. Power is proportional to the sample size, significance criterion (a level) and effect size, and is inversely proportional to the variance in the population. Effect size is a measure of biological significance: it is the difference between the results predicted by the null hypothesis and the actual state of the population being tested. In our example effect size is the difference in probability of predation between size classes. Power analysis can be used to determine whether the experiment had a good chance of producing a statistically significant result if a biologically significant difference existed in the population. In other words, whether the Correspondence: Len Thomas, Centre for Applied Conservation Biology, Faculty of Forestry, No. 270-2357 Main Mall, Vancouver, British Columbia V6T 1Z4, Canada (email: lthomas@unixg.ubc.ca).