Issues in biomedical statistics: comparing means by computer-intensive tests.

In this review there are described two alternatives to classical tests for distinguishing means. These are called computer-intensive because they can only be performed on fast computers. Permutation procedures have the virtue in that they are easy to understand, they can be employed to analyse small sets of experimental data, and under the randomization model of inference (though not the population model) they require no assumptions except that the experimental groups have been constructed by randomization. Bootstrap procedures are designed for use under the population model of inference (though not the randomization model) and are best suited to larger sets of experimental data. Non-parametric bootstrapping requires populations to be sampled randomly, but it depends on no prior assumptions about the distributions of those populations. It is argued that if randomization rather than random sampling has been done, permutation tests are superior to the classical t and F tests for detecting differences between means and therefore should replace them. If random sampling has been done, non-parametric bootstrap techniques may prove to be superior to classical tests for constructing population confidence intervals or testing hypotheses. However, their accuracy, especially for hypothesis-testing and when samples are small, has yet to be firmly established and there is a dearth of commercial software with which they can be executed on personal computers.