A boosting method with asymmetric mislabeling probabilities which depend on covariates

A new boosting method for a kind of noisy data is developed, where the probability of mislabeling depends on the label of a case. The mechanism of the model is based on a simple idea and gives natural interpretation as a mislabel model. The boosting algorithm is derived from an extension of the exponential loss function, which provides the AdaBoost algorithm. A connection between the proposed method and an asymmetric mislabel model is shown. It is also shown that the loss function proposed constructs a classifier which attains the minimum error rate for a true label. Numerical experiments illustrate how well the proposed method performs in comparison to existing methods.

[1]  John D. Lafferty,et al.  Boosting and Maximum Likelihood for Exponential Models , 2001, NIPS.

[2]  P. Lachenbruch.  Discriminant Analysis When the Initial Samples Are Misclassified , 1966 .

[3]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[4]  Hideo Suzuki,et al.  A ROBUST BOOSTING METHOD FOR MISLABELED DATA , 2004 .

[5]  Armin Shmilovici,et al.  Using a VOM model for reconstructing potential coding regions in EST sequences , 2007, Comput. Stat..

[6]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[7]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[8]  Takafumi Kanamori,et al.  Information Geometry of U-Boost and Bregman Divergence , 2004, Neural Computation.

[9]  Enrico Blanzieri,et al.  Detecting potential labeling errors in microarrays by data perturbation , 2006, Bioinform..

[10]  G. Lugosi,et al.  Complexity regularization via localized random penalties , 2004, math/0410091.

[11]  Consistency of Penalized Risk of Boosting Methods in Binary Classification , 2008 .

[12]  Peter L. Bartlett,et al.  AdaBoost is Consistent , 2006, J. Mach. Learn. Res..

[13]  Takafumi Kanamori,et al.  Robust Loss Functions for Boosting , 2007, Neural Computation.

[14]  A. Ekholm,et al.  A MODEL FOR A BINARY RESPONSE WITH MISCLASSIFICATIONS , 1982 .

[15]  Peter L. Bartlett,et al.  Boosting Algorithms as Gradient Descent in Function Space , 2007 .

[16]  Paul A. Viola,et al.  Fast and Robust Classification using Asymmetric AdaBoost and a Detector Cascade , 2001, NIPS.

[17]  François Fleuret.  Multi-layer boosting for pattern recognition , 2009, Pattern Recognit. Lett..

[18]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[19]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[20]  J. Copas.  Binary Regression Models for Contaminated Data , 1988 .

[21]  Shinto Eguchi,et al.  Robustifying AdaBoost by Adding the Naive Error Rate , 2004, Neural Computation.

[22]  R. Chhikara,et al.  Linear discriminant analysis with misallocation in training samples , 1984 .

[23]  David Mease,et al.  Boosted Classification Trees and Class Probability/Quantile Estimation , 2007, J. Mach. Learn. Res..

[24]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[25]  G. Lugosi,et al.  On the Bayes-risk consistency of regularized boosting methods , 2003 .

[26]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[27]  Osamu Watanabe,et al.  MadaBoost: A Modification of AdaBoost , 2000, COLT.