Evolving kernel functions for SVMs by genetic programming

hybrid model for evolving support vector machine (SVM) kernel functions is developed in this paper. The kernel expression is considered as a parameter of the SVM algorithm and the current approach tries to find the best expression for this SVM parameter. The model is a hybrid technique that combines a genetic programming (GP) algorithm and a support vector machine (SVM) algorithm. Each GP chromosome is a tree encoding the mathematical expression for the kernel function. The evolved kernel is compared to several human-designed kernels and to a previous genetic kernel on several datasets. Numerical experiments show that the SVM embedding our evolved kernel performs statistically better than standard kernels, but also than previous genetic kernel for all considered classification problems.

[1]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[2]  R. Calvo,et al.  Neural Network Prediction of Solar Activity , 1995 .

[3]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[4]  Mihai Oltean,et al.  Evolving Crossover Operators for Function Optimization , 2006, EuroGP.

[5]  Michel Verleysen,et al.  Time series forecasting: Obtaining long term trends with self-organizing maps , 2005, Pattern Recognit. Lett..

[6]  D. Hathaway,et al.  The shape of the sunspot cycle , 1994 .

[7]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[8]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[9]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[10]  Michael G. Madden,et al.  The Genetic Kernel Support Vector Machine: Description and Evaluation , 2005, Artificial Intelligence Review.

[11]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[12]  Ah Chung Tsoi,et al.  Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference , 2001, Machine Learning.

[13]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[14]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[15]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.