Non-uniform Layered Clustering for Ensemble Classifier Generation and Optimality

In this paper we present an approach to generate ensemble of classifiers using non-uniform layered clustering. In the proposed approach the dataset is partitioned into variable number of clusters at different layers. A set of base classifiers is trained on the clusters at different layers. The decision on a pattern at each layer is obtained from the classifier trained on the nearest cluster and the decisions from the different layers are fused using majority voting to obtain the final verdict. The proposed approach provides a mechanism to obtain the optimal number of layers and clusters using a Genetic Algorithm. Clustering identifies difficult-to-classify patterns and layered non-uniform clustering approach brings in diversity among the base classifiers at different layers. The proposed method performs relatively better than the other state-of-art ensemble classifier generation methods as evidenced from the experimental results.

[1]  Mario Köppen,et al.  Advances in Neuro-Information Processing, 15th International Conference, ICONIP 2008, Auckland, New Zealand, November 25-28, 2008, Revised Selected Papers, Part I , 2009, International Conference on Neural Information Processing.

[2]  Ludmila I. Kuncheva,et al.  Switching between selection and fusion in combining classifiers: an experiment , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[3]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[4]  Grigorios Tsoumakas,et al.  Clustering based multi-label classification for image annotation and retrieval , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[5]  Tim W. Nattkemper,et al.  The Diversity of Regression Ensembles Combining Bagging and Random Subspace Method , 2008, ICONIP.

[6]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[7]  Xin Yao,et al.  An analysis of diversity measures , 2006, Machine Learning.

[8]  André L. V. Coelho,et al.  Ensembling Heterogeneous Learning Models with Boosting , 2009, ICONIP.

[9]  Loris Nanni,et al.  FuzzyBagging: A novel ensemble of classifiers , 2006, Pattern Recognit..

[10]  Stefan Schliebs,et al.  Initialization Dependence of Clustering Algorithms , 2008, ICONIP.

[11]  Mohamed S. Kamel,et al.  A generalized adaptive ensemble generation and aggregation approach for multiple classifier systems , 2009, Pattern Recognit..

[12]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[13]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[14]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[15]  Lior Rokach,et al.  Space Decomposition in Data Mining: A Clustering Approach , 2002, ISMIS.

[16]  Robi Polikar,et al.  Learn$^{++}$ .NC: Combining Ensemble of Classifiers With Dynamically Weighted Consult-and-Vote for Efficient Incremental Learning of New Classes , 2009, IEEE Transactions on Neural Networks.