The State of Boosting ∗

In many problem domains, combining the predictions of several models often results in a model with improved predictive performance. Boosting is one such method that has shown great promise. On the applied side, empirical studies have shown that combining models using boosting methods produces more accurate classification and regression models. These methods are extendible to the exponential family as well as proportional hazards regression models. This article shows that boosting, which is still new to statistics, is widely applicable. I will introduce boosting, discuss the current state of boosting, and show how these methods connect to more standard statistical practice.

[1]  P. J. Huber.  Robust Estimation of a Location Parameter , 1964 .

[2]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[3]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[4]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[5]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[6]  Robert Tibshirani,et al.  Model Search and Inference By Bootstrap "bumping , 1995 .

[7]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[8]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[9]  Harris Drucker,et al.  Improving Regressors using Boosting Techniques , 1997, ICML.

[10]  Visualizing the Simple Bayesian Classi er , 1997 .

[11]  L. Breiman.  Arcing Classifiers , 1998 .

[12]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Robert E. Schapire,et al.  A Brief Introduction to Boosting , 1999, IJCAI.

[15]  Peter L. Bartlett,et al.  Boosting Algorithms as Gradient Descent , 1999, NIPS.

[16]  Robert E. Schapire,et al.  Theoretical Views of Boosting , 1999, EuroCOLT.

[17]  Leo Breiman,et al.  Prediction Games and Arcing Algorithms , 1999, Neural Computation.

[18]  Thomas Richardson,et al.  Boosting methodology for regression problems , 1999, AISTATS.

[19]  Thomas S. Richardson,et al.  Generalization of boosting algorithms and applications of bayesian inference for massive datasets , 1999 .

[20]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[21]  D.,et al.  Regression Models and Life-Tables , 2022 .