Genetic fuzzy classification fusion of multiple SVMs for biomedical data

Classification of biomedical data faces a special challenge because of the characteristics of the data: too few data examples with too many features. How to improve the classification performance or the generalization ability of a classifier in the biomedical domain becomes one of the active research areas. One approach is to build a fusion model to combine multiple classifiers together and result in a combined classifier which can achieve a better performance than any of its composing individual classifiers. In this paper, we propose a SVM classifier fusion model to combine multiple SVMs by applying the knowledge of fuzzy logic and genetic algorithms. The fuzzy logic system (FLS) is constructed based on SVM accuracies and distances of data examples to SVM hyperplanes in SVM feature spaces. A genetic algorithm (GA) is used to tune the fuzzy membership functions (MFs) in the FLS and determine the optimal fuzzy fusion model. We have applied the proposed model to two biomedical data: colon tumor data and ovarian cancer data. Our experiment shows that multiple SVM classifiers complement each other well in the proposed fusion model and the ensemble achieves a better, more robust and more reliable performance than individual composing SVMs.

[1]  Francisco Herrera,et al.  GENERATING FUZZY RULES FROM EXAMPLES USING GENETIC ALGORITHMS , 1995 .

[2]  Abdollah Homaifar,et al.  Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[3]  Thomas Bäck,et al.  A Survey of Evolution Strategies , 1991, ICGA.

[4]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[8]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[9]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[10]  Francisco Herrera,et al.  Ten years of genetic fuzzy systems: current framework and new trends , 2004, Fuzzy Sets Syst..

[11]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[12]  G. Langholz,et al.  Genetic-Based New Fuzzy Reasoning Models with Application to Fuzzy Control , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[13]  Francisco Herrera,et al.  A three-stage evolutionary process for learning descriptive and approximate fuzzy-logic-controller knowledge bases from examples , 1997, Int. J. Approx. Reason..

[14]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Patrick P. K. Chan,et al.  Fuzzy support vector machines for solving two-class problems , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[16]  H. P. Huang,et al.  Fuzzy Support Vector Machines for Pattern Recognition and Data Mining , 2002 .

[17]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[18]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[19]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  Chyck Karr,et al.  Applying genetics to fuzzy logic , 1991 .