A novel genetic algorithm approach for simultaneous feature and classifier selection in multi classifier system

In this paper we introduce a novel approach for classifier and feature selection in a multi-classifier system using Genetic Algorithm (GA). Specifically, we propose a 2-part structure for each chromosome in which the first part is encoding for classifier and the second part is encoding for feature. Our structure is simple in the implementation of the crossover as well as the mutation stage of GA. We also study 8 different fitness functions for our GA based algorithm to explore the optimal fitness functions for our model. Experiments are conducted on both 14 UCI Machine Learning Repository and CLEF2009 medical image database to demonstrate the benefit of our model on reducing classification error rate.

[1]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[2]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[3]  Lakhmi C. Jain,et al.  Designing classifier fusion systems by genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[4]  Bogdan Gabrys,et al.  Genetic algorithms in classifier fusion , 2006, Appl. Soft Comput..

[5]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Seong-Hoon Kim,et al.  X-ray Image Classification Using Random Forests with Local Wavelet-Based CS-Local Binary Patterns , 2011, Journal of Digital Imaging.

[7]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ian H. Witten,et al.  Issues in Stacked Generalization , 2011, J. Artif. Intell. Res..

[9]  Loris Nanni,et al.  A genetic encoding approach for learning methods for combining classifiers , 2009, Expert Syst. Appl..

[10]  Christopher J. Merz,et al.  Using Correspondence Analysis to Combine Classifiers , 1999, Machine Learning.

[11]  Ludmila I. Kuncheva,et al.  A Theoretical Study on Six Classifier Fusion Strategies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  James C. Bezdek,et al.  Decision templates for multiple classifier fusion: an experimental comparison , 2001, Pattern Recognit..

[13]  Alexander K. Seewald,et al.  How to Make Stacking Better and Faster While Also Taking Care of an Unknown Weakness , 2002, International Conference on Machine Learning.

[14]  Hakan Erdogan,et al.  Linear classifier combination and selection using group sparse regularization and hinge loss , 2013, Pattern Recognit. Lett..

[15]  Lior Rokach,et al.  Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography , 2009, Comput. Stat. Data Anal..