A Comparison of Three Voting Methods for Bagging with the MLEM2 Algorithm

This paper presents results of experiments on some data sets using bagging on the MLEM2 rule induction algorithm. Three different methods of ensemble voting, based on support (a non-democratic voting in which ensembles vote with their strengths), strength only (an ensemble with the largest strength decides to which concept a case belongs) and democratic voting (each ensemble has at most one vote) were used. Our conclusions are that though in most cases democratic voting was the best, it is not significantly better than voting based on support. The strength voting was the worst voting method.

[1]  Z. Pawlak.  Rough Sets: Theoretical Aspects of Reasoning about Data , 1991 .

[2]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[3]  Adam Weintrit,et al.  Methods and Algorithms , 2011 .

[4]  Jerzy Stefanowski,et al.  On Combined Classifiers, Rule Induction and Rough Sets , 2007, Trans. Rough Sets.

[5]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[6]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[7]  Lawrence O. Hall,et al.  Comparing pure parallel ensemble creation techniques against bagging , 2003, Third IEEE International Conference on Data Mining.

[8]  Jerzy Stefanowski,et al.  Ensembles of Abstaining Classifiers Based on Rule Sets , 2009, ISMIS.

[9]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[10]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[11]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[12]  Jerzy W. Grzymala-Busse,et al.  A New Version of the Rule Induction System LERS , 1997, Fundam. Informaticae.

[13]  João Gama,et al.  Combining Classifiers by Constructive Induction , 1998, ECML.

[14]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[15]  Bernard Zenko,et al.  A comparison of stacking with meta decision trees to bagging, boosting, and stacking with other methods , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[16]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[17]  G DietterichThomas.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees , 2000 .

[18]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[19]  Jerzy Stefanowski,et al.  The Bagging and n2-Classifiers Based on Rules Induced by MODLEM , 2004, Rough Sets and Current Trends in Computing.

[20]  Ljupčo Todorovski,et al.  A comparison of stacking with MDTs to bagging, boosting, and other stacking methods , 2001 .

[21]  Jerzy W. Grzymala-Busse,et al.  Global discretization of continuous attributes as preprocessing for machine learning , 1996, Int. J. Approx. Reason..