An Evolutionary and Attribute-Oriented Ensemble Classifier

In the research area of decision tree, numerous researchers have been focusing on improving the predictive accuracy. However, obvious improvement can hardly be made until the introduction of the ensemble classifier. In this paper, we propose an Evolutionary Attribute-Oriented Ensemble Classifier (EAOEC) to improve the accuracy of sub-classifiers and at the same time maintain the diversity among them. EAOEC uses the idea of evolution to choose proper attribute subset for the building of every sub-classifier. To avoid the huge computation cost for the evolution, EAOEC uses the gini value gained during the construction of a sub-tree as the evolution basis to build the next sub-tree. Eventually, EAOEC classifier uses uniform weight voting to combine all sub-classifiers and experiments show that EAOEC can efficiently improve the predictive accuracy.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  Jorma Rissanen,et al.  MDL-Based Decision Tree Pruning , 1995, KDD.

[3]  Enric Plaza,et al.  Machine Learning: ECML 2000 , 2003, Lecture Notes in Computer Science.

[4]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Ludmila I. Kuncheva,et al.  Relationships between combination methods and measures of diversity in combining classifiers , 2002, Inf. Fusion.

[6]  Jorma Rissanen,et al.  SLIQ: A Fast Scalable Classifier for Data Mining , 1996, EDBT.

[7]  Robert P. W. Duin,et al.  Is independence good for combining classifiers? , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[8]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[9]  Lior Rokach,et al.  Ensemble Methods for Classifiers , 2005, The Data Mining and Knowledge Discovery Handbook.

[10]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[11]  Raymond J. Mooney,et al.  Diverse ensembles for active learning , 2004, ICML.

[12]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[13]  David H. Wolpert,et al.  An Efficient Method To Estimate Bagging's Generalization Error , 1999, Machine Learning.

[14]  Padraig Cunningham,et al.  Diversity versus Quality in Classification Ensembles Based on Feature Selection , 2000, ECML.

[15]  L. Darrell Whitley,et al.  Genetic Approach to Feature Selection for Ensemble Creation , 1999, GECCO.

[16]  Terry Windeatt.  Diversity/accuracy and ensemble classifier design , 2004, ICPR 2004.

[17]  Kyuseok Shim,et al.  PUBLIC: A Decision Tree Classifier that Integrates Building and Pruning , 1998, Data Mining and Knowledge Discovery.

[18]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[19]  Qing Zhang,et al.  Enhancing SNNB with Local Accuracy Estimation and Ensemble Techniques , 2005, DASFAA.

[20]  David W. Opitz,et al.  Feature Selection for Ensembles , 1999, AAAI/IAAI.