Does Boosting Over t: Views From an Exact Solution

We consider the AdaBoost algorithm using the piecewise constant base hypotheses on the predictor space [0; 1]. The boosted solutions are not unique, and one exact solution after a su ciently large number of rounds, is shown to generate the nearest neighbor rule. Asymptotic result for the prediction error is provided for piecewise Lipshitz signals, which illustrates that the AdaBoost algorithm in this case does over t, but the amount of over t is usually small. In fact, in the case when the labels are noiseless but the predictor / example is random, we show that the boosted procedure is nearly optimal (within a logarithm factor to the asymptotic minimax rate). Index Terms| Bayes rule, boosting, consistency, convergence rate, minimax rate, nearest neighbor rule, over tting, prediction error.