Multi-scale Support Vector Machine Optimization by Kernel Target-Alignment

The problem considered is the optimization of a multi-scale kernel, where a different width is chosen for each feature. This idea has been barely studied in the literature, and through the use of evolutionary or gradient descent approaches, which explicitly train the learning machine and thereby incur high computacional cost. To cope with this limitation, the problem is explored by making use of an analytical methodology known as kernel-target alignment, where the kernel is optimized by aligning it to the so-called ideal kernel matrix. The results show that the proposal leads to better performance and simpler models at limited computational cost when applying the binary Support Vector Machine (SVM) paradigm.

[1]  Boonserm Kijsirikul,et al.  Evolutionary strategies for multi-scale radial basis function kernels in support vector machines , 2005, GECCO '05.

[2]  Melanie Hilario,et al.  Margin and Radius Based Multiple Kernel Learning , 2009, ECML/PKDD.

[3]  Carlos Soares,et al.  Selecting parameters of SVM using meta-learning and kernel matrix-based meta-features , 2006, SAC '06.

[4]  Ivor W. Tsang,et al.  Learning with Idealized Kernels , 2003, ICML.

[5]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[6]  Sancho Salcedo-Sanz,et al.  Multi-parametric Gaussian Kernel Function Optimization for ε-SVMr Using a Genetic Algorithm , 2011, IWANN.

[7]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[8]  Marco Cuturi,et al.  Fast Global Alignment Kernels , 2011, ICML.

[9]  Mehryar Mohri,et al.  Algorithms for Learning Kernels Based on Centered Alignment , 2012, J. Mach. Learn. Res..

[10]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[11]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[12]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[13]  A. Sowmya,et al.  The anisotropic Gaussian kernel for SVM classification of HRCT images of the lung , 2004, Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004..

[14]  Laura Diosan,et al.  Improving classification performance of Support Vector Machine by genetically optimising kernel shape and hyper-parameters , 2010, Applied Intelligence.