Evolutionary strategies for multi-scale radial basis function kernels in support vector machines

In support vector machines (SVM), the kernel functions which compute dot product in feature space significantly affect the performance of classifiers. Each kernel function is suitable for some tasks. A universal kernel is not possible, and the kernel must be chosen for the tasks under consideration by hand. In order to obtain a flexible kernel function, a family of radial basis function (RBF) kernels is proposed. Multi-scale RBF kernels are combined by including weights. Then, the evolutionary strategies are used to adjust these weights and the widths of the RBF kernels. The proposed kernel is proved to be a Mercer's kernel. The experimental results show that the use of multi-scale RBF kernels result in better performance than that of a single Gaussian RBF on benchmarks.

[1]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  Holger Frohlich,et al.  Feature Selection for Support Vector Machines by Means of Genetic Algorithms -Diploma Thesis in Computer Science- , 2002 .

[4]  Christian Igel,et al.  Evolutionary tuning of multiple SVM parameters , 2005, ESANN.

[5]  Ronald M. Summers,et al.  Feature selection for computer-aided polyp detection using genetic algorithms , 2003, SPIE Medical Imaging.

[6]  Christian Igel,et al.  Multi-objective Model Selection for Support Vector Machines , 2005, EMO.

[7]  Ching Y. Suen,et al.  KMOD - a new support vector machine kernel with moderate decreasing for pattern recognition. Application to digit image recognition , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[8]  Elena Marchiori,et al.  Feature selection in proteomic pattern data with support vector machines , 2004, 2004 Symposium on Computational Intelligence in Bioinformatics and Computational Biology.

[9]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[10]  Thomas Philip Runarsson,et al.  Asynchronous Parallel Evolutionary Model Selection for Support Vector Machines , 2004 .

[11]  Guido Smits,et al.  Improved SVM regression using mixtures of kernels , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[12]  S. Gunn.  Support Vector Machines for Classification and Regression , 1998 .

[13]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[14]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[15]  Simon J. Perkins,et al.  Genetic Algorithms and Support Vector Machines for Time Series Classification , 2002, Optics + Photonics.

[16]  Federico Girosi,et al.  Support Vector Machines: Training and Applications , 1997 .

[17]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[18]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[19]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[20]  Bernhard Schölkopf,et al.  Feature selection for support vector machines by means of genetic algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[21]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[22]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[23]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[24]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[25]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[26]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[27]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[28]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[29]  Garrison W. Greenwood,et al.  Adaptive integration using evolutionary strategies , 1996, Proceedings of 3rd International Conference on High Performance Computing (HiPC).

[30]  Elena Marchiori,et al.  Analysis of Proteomic Pattern Data for Cancer Detection , 2004, EvoWorkshops.