On bootstrap resampling and iteration

SUMMARY We propose a single unifying approach to bootstrap resampling, applicable to a very wide range of statistical problems. It enables attention to be focused sharply on one or more characteristics which are of major importance in any particular problem, such as coverage error or length for confidence intervals, or bias for point estimation. Our approach leads easily and directly to a very general form of bootstrap iteration, unifying and generalizing present disparate accounts of this subject. It also provides simple solutions to relatively complex problems, such as a suggestion by Lehmann (1986) for 'conditionally' short confidence intervals. We set out a single unifying principle guiding the operation of bootstrap resampling, applicable to a very wide range of statistical problems including bias reduction, shrinkage, hypothesis testing and confidence interval construction. Our principle differs from other approaches in that it focuses attention directly on a measure of quality or accuracy, expressed in the form of an equation whose solution is sought. A very general form of bootstrap iteration is an immediate consequence of iterating the empirical solution to this equation so as to improve accuracy. When employed for bias reduction, iteration of the resampling principle yields a competitor to the generalized jackknife, enabling bias to be reduced to arbitrarily low levels. When applied to confidence intervals it produces the techniques of Hall (1986) and Beran (1987). The resampling principle leads easily to solutions of new, complex problems, such as empirical versions of confidence intervals proposed by Lehmann (1986). Lehmann argued that an 'ideal' confidence interval is one which is short when it covers the true parameter value but not necessarily otherwise. The resampling principle suggests a simple empirical means of constructing such intervals. Section 2 describes the general principle, and ? 3 shows how it leads naturally to bootstrap iteration. There we show that in many problems of practical interest, such as bias reduction and coverage-error reduction in two-sided confidence intervals, each iteration reduces error by the factor n-1, where n is sample size. In the case of confidence intervals our result sharpens one of Beran (1987), who showed that coverage error is reduced by the factor n-2 in two-sided intervals. The main exception to our n-1 rule is coverage error of one-sided intervals, where error is reduced by the factor n-A at each iteration. Our approach to bootstrap iteration serves to unify not just the philosophy of iteration for different statistical problems, but also different techniques of iteration for the same