Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By

The main and important contribution of this paper is in establishing a connection between boosting, a newcomer to the statistics scene, and additive models. One of the main properties of boosting that has made it interesting to statisticians and others is its relative (but not complete) immunity to overrtting. As pointed out by the authors, the current paper does not address this issue. Leo Breiman 1] tried to explain this behaviour in terms of bias and variance. In our paper with Bartlett and Lee 4], we gave an explanation in terms of the \margins" of the training examples and the VC-dimension of the base class. Breiman, as well as the current paper, point out that our bounds are very rough and yield bounds that are not useful in practice. While this is clearly true at this time, it is also true that the analysis given by Breiman and by this paper yield no provable bounds whatsoever. It is completely unclear whether this analysis can be used to predict the performance of classiication rules outside of the training sample. At the root of this argument about boosting is a much more fundamental argument about the type of prior assumptions that one should make when embarking on the task of inducing a classiication rule from data. The assumption that seems to underlie the use of maximum likelihood in the 1