Evolving Committees of Support Vector Machines

The main emphasis of the technique developed in this work for evolving committees of support vector machines (SVM) is on a two phase procedure to select salient features. In the first phase, clearly redundant features are eliminated based on the paired t-test comparing the SVM output sensitivity-based saliency of the candidate and the noise feature. In the second phase, the genetic search integrating the steps of training, aggregation of committee members, and hyper-parameter as well as feature selection into the same learning process is employed. A small number of genetic iterations needed to find a solution is the characteristic feature of the genetic search procedure developed. The experimental tests performed on five real world problems have shown that significant improvements in correct classification rate can be obtained in a small number of iterations if compared to the case of using all the features available.

[1]  Antanas Verikas,et al.  Integrating Global and Local Analysis of Color, Texture and Geometrical Information for Categorizing Laryngeal Images , 2006, Int. J. Pattern Recognit. Artif. Intell..

[2]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[3]  Paul D. Gader,et al.  Fusion of handwritten word classifiers , 1996, Pattern Recognit. Lett..

[4]  Jacek M. Zurada,et al.  Advances in Neural Networks - ISNN 2006, Third International Symposium on Neural Networks, Chengdu, China, May 28 - June 1, 2006, Proceedings, Part I , 2006, International Symposium on Neural Networks.

[5]  Kenneth W. Bauer,et al.  Improved feature screening in feedforward neural networks , 1996, Neurocomputing.

[6]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[7]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[8]  Hongbin Zhang,et al.  Feature selection using tabu search method , 2002, Pattern Recognit..

[9]  Antanas Verikas,et al.  Selecting salient features for classification based on neural network committees , 2004, Pattern Recognit. Lett..

[10]  Steven K. Rogers,et al.  Bayesian selection of important features for feedforward neural networks , 1993, Neurocomputing.

[11]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[12]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[13]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[14]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[15]  Antanas Verikas,et al.  Selecting Variables for Neural Network Committees , 2006, ISNN.

[16]  M. Bacauskiene,et al.  Multiple feature sets based categorization of laryngeal images , 2007, Comput. Methods Programs Biomed..

[17]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[18]  Paul Scheunders,et al.  Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery , 2002, Pattern Recognit. Lett..

[19]  Antanas Verikas,et al.  Feature selection with neural networks , 2002, Pattern Recognit. Lett..

[20]  K.Z. Mao,et al.  Orthogonal forward selection and backward elimination algorithms for feature subset selection , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Alexey Tsymbal,et al.  Ensemble feature selection with the simple Bayesian classification , 2003, Inf. Fusion.

[22]  Antanas Verikas,et al.  Soft combination of neural classifiers: A comparative study , 1999, Pattern Recognit. Lett..

[23]  Nurettin Acir,et al.  Automatic recognition of sleep spindles in EEG via radial basis support vector machine based on a modified feature selection algorithm , 2004, Neural Computing & Applications.

[24]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Hak-Keung Lam,et al.  Application of a modified neural fuzzy network and an improved genetic algorithm to speech recognition , 2007, Neural Computing and Applications.

[26]  Cheng-Lin Liu,et al.  Classifier combination based on confidence transformation , 2005, Pattern Recognit..

[27]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[28]  Tomaso A. Poggio,et al.  Image Representations and Feature Selection for Multimedia Database Search , 2003, IEEE Trans. Knowl. Data Eng..