A ShortIntroductionto Boosting Yoav Freund RobertE . Schapire AT & T Labs

Boostingis a generalmethodfor improving theaccuracy of any givenlearningalgorithm. This shortoverview paperintroducestheboostingalgorithmAdaBoost,andexplainstheunderlying theoryof boosting,including an explanationof why boostingoften doesnot suffer from overfittingaswell asboosting’ s relationshipto support-vectormachines.Someexamples of recentapplicationsof boostingarealsodescribed.

[1]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[2]  Y. Freund,et al.  Adaptive game playing using multiplicative weights , 1999 .

[3]  Yoshua Bengio,et al.  Training Methods for Adaptive Boosting of Neural Networks , 1997, NIPS.

[4]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[5]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[6]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[7]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[8]  Mark Craven,et al.  Learning Sparse Perceptrons , 1995, NIPS.

[9]  David W. Opitz,et al.  An Empirical Evaluation of Bagging and Boosting , 1997, AAAI/IAAI.

[10]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[11]  Yoram Singer,et al.  Boosting Applied to Tagging and PP Attachment , 1999, EMNLP.

[12]  Peter L. Bartlett,et al.  Direct Optimization of Margins Improves Generalization in Combined Classifiers , 1998, NIPS.

[13]  Dale Schuurmans,et al.  Boosting in the Limit: Maximizing the Margin of Learned Ensembles , 1998, AAAI/IAAI.

[14]  L. Breiman.  Arcing the edge , 1997 .

[15]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[16]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[17]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[18]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[19]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[20]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[21]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[22]  Harris Drucker,et al.  Boosting Performance in Neural Networks , 1993, Int. J. Pattern Recognit. Artif. Intell..

[23]  Yoav Freund,et al.  Game theory, on-line prediction and boosting , 1996, COLT '96.

[24]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[25]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[26]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[27]  Corinna Cortes,et al.  Boosting Decision Trees , 1995, NIPS.

[28]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[29]  Yoav Freund,et al.  An Adaptive Version of the Boost by Majority Algorithm , 1999, COLT '99.

[30]  Robert E. Schapire,et al.  Using output codes to boost multiclass learning problems , 1997, ICML.

[31]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[32]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[33]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[34]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[35]  Johannes Fürnkranz,et al.  Incremental Reduced Error Pruning , 1994, ICML.

[36]  Yoram Singer,et al.  Boosting and Rocchio applied to text filtering , 1998, SIGIR '98.

[37]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.