A ROBUST BOOSTING BY USING AN ADAPTIVE WEIGHT SCHEME

In the real world, it is extremely difficult to avoid errors; for instance, a doctor may misdiagnose patients. In other words, databases are never free from data entry or other related errors, and many kinds of mistakes are unavoidable in real world data sets. In existing approaches to pattern recognition, handling noisy data in the learning process always produces better generalization performance than if the noise were ignored. In this article, a novel and adaptive weighting mechanism for noise learning tasks is proposed, especially for boosting learning approaches, preventing the algorithm from concentrating on unreasonably noisy learning samples. Several experiments on UC Irvine Machine Learning Repository and a facial expression data set demonstrate the effectiveness of our method.

[1]  H. Schwenk,et al.  Adaboosting neural networks , 1997 .

[2]  Yongdai Kim.  Averaged Boosting: A Noise-Robust Ensemble Method , 2003, PAKDD.

[3]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[4]  Matti Pietikäinen,et al.  A Coarse-to-Fine Classification Scheme for Facial Expression Recognition , 2004, ICIAR.

[5]  Harris Drucker,et al.  Learning algorithms for classification: A comparison on handwritten digit recognition , 1995 .

[6]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[7]  L. Breiman.  Arcing classifier (with discussion and a rejoinder by the author) , 1998 .

[8]  L. Breiman.  Arcing Classifiers , 1998 .

[9]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[10]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[11]  Osamu Watanabe,et al.  MadaBoost: A Modification of AdaBoost , 2000, COLT.

[12]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[13]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[14]  Peter L. Bartlett,et al.  Boosting Algorithms as Gradient Descent , 1999, NIPS.

[15]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[16]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[17]  Anneleen Van Assche,et al.  Ensemble Methods for Noise Elimination in Classification Problems , 2003, Multiple Classifier Systems.

[18]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[19]  J. Ross Quinlan,et al.  Boosting First-Order Learning , 1996, ALT.

[20]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[21]  Xiaoyi Feng.  Facial expression recognition based on local binary patterns and coarse-to-fine classification , 2004, The Fourth International Conference onComputer and Information Technology, 2004. CIT '04..

[22]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[23]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.