Analysis of Diversity Assurance Methods for Combined Classifiers

Assuring diversity of classifiers in an ensemble plays a crucial role in the multiple classifier system design. The paper presents a comparative study of selected methods which can assure the diversity by manipulating the individual classifier inputs i.e., they train learner using subspaces of a feature set or they try to exploit local competencies of individual classifier for a given subset of feature space. This work is a starting point for developing new methods of diversity assurance embedded in a multiple classifier system design. All methods had been evaluated on the basis of computer experiments which were carried out on benchmark datasets. On the basis of received results conclusions about the usefulness of examined methods for certain types of problems were drawn.

[1]  Ludmila I. Kuncheva,et al.  Clustering-and-selection model for classifier combination , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[2]  Mario Vento,et al.  A Cascaded Multiple Expert System for Verification , 2000, Multiple Classifier Systems.

[3]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[4]  Fabio Roli,et al.  Multiple Classifier Systems, 9th International Workshop, MCS 2010, Cairo, Egypt, April 7-9, 2010. Proceedings , 2010, MCS.

[5]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Bartosz Krawczyk,et al.  Classifier Committee Based on Feature Selection Method for Obstructive Nephropathy Diagnosis , 2011 .

[8]  Derek Partridge,et al.  Software Diversity: Practical Statistics for Its Measurement and Exploitation | Draft Currently under Revision , 1996 .

[9]  Kurt Hornik,et al.  Open-source machine learning: R meets Weka , 2009, Comput. Stat..

[10]  Fabio Roli,et al.  Design of effective multiple classifier systems by clustering of classifiers , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[11]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Michal Wozniak,et al.  Algorithm of designing compound recognition system on the basis of combining classifiers with simultaneous splitting feature space into competence areas , 2009, Pattern Analysis and Applications.

[13]  D. Wolpert The Supervised Learning No-Free-Lunch Theorems , 2002 .

[14]  Ian H. Witten,et al.  WEKA: a machine learning workbench , 1994, Proceedings of ANZIIS '94 - Australian New Zealnd Intelligent Information Systems Conference.

[15]  Ethem Alpaydın,et al.  Combined 5 x 2 cv F Test for Comparing Supervised Classification Learning Algorithms , 1999, Neural Comput..

[16]  Bogdan Gabrys,et al.  Classifier selection for majority voting , 2005, Inf. Fusion.

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[19]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[20]  Chao-Fu Hong,et al.  Semantic Methods for Knowledge Management and Communication , 2011 .