Localized Boosting

We introduce and analyze LocBoost, a new boosting algorithm, which leads to the incremental construction of a mixture of experts type architecture. We provide upper bounds on the expected loss of such models in terms of the smoothness properties of the gating functions appearing in the mixture of experts model. Furthermore, an incremental algorithm is proposed for the construction of the classifier, based on a maximum-likelihood approach and the EM algorithm. Preliminary numerical results appear to be promising.

[1]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[2]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[3]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[4]  Eddy Mayoraz,et al.  DynaBoost: Combining Boosted Hypotheses in a Dynamic Way , 1999 .

[5]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[6]  Y. Freund,et al.  Discussion of the Paper \additive Logistic Regression: a Statistical View of Boosting" By , 2000 .

[7]  M. Schervish.  Theory of Statistics , 1995 .

[8]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[9]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[10]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[11]  F. Y. Edgeworth,et al.  The theory of statistics , 1996 .

[12]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[13]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[14]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[15]  Yuhong Yang,et al.  Minimax Nonparametric Classification—Part I: Rates of Convergence , 1998 .

[16]  Tom,et al.  A simple cost function for boostingMarcus , .

[17]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[18]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[19]  A. Barron,et al.  Estimation of mixture models , 1999 .

[20]  L. Breiman.  Arcing the edge , 1997 .

[21]  Peter L. Bartlett,et al.  Functional Gradient Techniques for Combining Hypotheses , 2000 .

[22]  Gunnar Rätsch,et al.  Regularizing AdaBoost , 1998, NIPS.

[23]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.