Heuristics for Kernels Adaptation in Support Vector Machines

Support Vector Machines are an algorithm introduced by Vapnik and coworkers [9], [10]. They are based on the idea that if input points are mapped to a high dimensional feature space then, a separating hyperplane can be easily found. SVM and kernel methods have been applied to a wide class of problems including approximation and classification and they have proven a remarkable performance on real world problems. An important step in their design is the setting of the kernels parameters which defines the structure of the high dimensional feature space where a maximal margin hyperplane will be found. Too rich feature space, e.g. small kernel parameters, will over-fit the data and hence result in a poor generalisation error, whereas if the kernel parameter is too big, the model will not be able to separate the data. In this paper we firstly propose a heuristic that permits the individual control of the growth in each kernel, which results in more sparse models with higher prediction accuracy. Secondly, a heuristic resulting from the combination of SVM trained by linear programming (LP) and EC for the optimisation of the kernels width is proposed.