Comparison of single and ensemble classifiers in terms of accuracy and execution time

Classification accuracy and execution time are two important parameters in the selection of classification algorithms. In our experiments, 12 different ensemble algorithms, and 11 single classifiers are compared according to their accuracies and train/test time over 36 datasets. The results show that Rotation Forest has the highest accuracy. However, when accuracy and execution time are considered together, Random Forest and Random Committees can be the best choices.

[1]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[2]  Haijia Shi.  Best-first Decision Tree Learning , 2007 .

[3]  Ian H. Witten,et al.  Stacking Bagged and Dagged Models , 1997, ICML.

[4]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[5]  Lawrence O. Hall,et al.  A Comparison of Decision Tree Ensemble Creation Techniques , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ethem Alpaydın,et al.  Combined 5 x 2 cv F Test for Comparing Supervised Classification Learning Algorithms , 1999, Neural Comput..

[7]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[10]  Eibe Frank,et al.  Logistic Model Trees , 2003, Machine Learning.

[11]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[12]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[13]  Raymond J. Mooney,et al.  Constructing Diverse Classifier Ensembles using Artificial Training Examples , 2003, IJCAI.

[14]  Lawrence O. Hall,et al.  A Comparison of Ensemble Creation Techniques , 2004, Multiple Classifier Systems.

[15]  Geoffrey I. Webb,et al.  MultiBoosting: A Technique for Combining Boosting and Wagging , 2000, Machine Learning.

[16]  D. Kibler,et al.  Instance-based learning algorithms , 2004, Machine Learning.

[17]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[18]  Ian Witten,et al.  Data Mining , 2000 .

[19]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[20]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[23]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[24]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[25]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[26]  Stefan Kramer,et al.  Ensembles of Balanced Nested Dichotomies for Multi-class Problems , 2005, PKDD.

[27]  Geoff Holmes,et al.  Multiclass Alternating Decision Trees , 2002, ECML.