Performance evaluation of kernel functions based on grid search for support vector regression

The support vector machine (SVM) has recently been successfully used for classification and regression problems. As the core of SVM, the kernel function is used to map the input vectors into a higher dimensional feature space to realize the non-linear algorithm for original space. The selection of kernel function and corresponding parameters directly affects the learning ability and generalization ability of the support vector regression (SVR). This paper compares the performance of different kernel functions on the energy efficiency data obtained from the UCI repository. The kernel functions used are as follows: Linear, Gaussian, Polynomial, Sigmoid, Inv. Multiquadric and the Semi Local kernel. The influence of parameters on the mean square error of cross validation (MSECV) for each kernel is shown in the paper. The combination of SVR parameters and kernel parameter with minimum MSECV obtained by grid search method based on cross-validation is chosen for SVR model building. The experiment results based on the dataset showed that the Semi Local kernel has the best predictive capability. The mean-squared error of prediction (MSEP) of the Semi Local kernel was 97.13%, 8.26%, 0.96%, 99.95%, 27.36% lower than the value of the Linear, Gaussian, Polynomial, Sigmoid and Inv. Multiquadric kernel.

[1]  Li Lanlan,et al.  Support vector regression and ant colony optimization for HVAC cooling load prediction , 2010, 2010 International Symposium on Computer, Communication, Control and Automation (3CA).

[2]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[3]  Helena Geirinhas Ramos,et al.  An SVM approach with electromagnetic methods to assess metal plate thickness , 2014 .

[4]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[5]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Thorsten Joachims,et al.  Estimating the Generalization Performance of an SVM Efficiently , 2000, ICML.

[8]  Alexander J. Smola,et al.  Support Vector Machine Reference Manual , 1998 .

[9]  I. Babuska,et al.  The partition of unity finite element method: Basic theory and applications , 1996 .

[10]  Athanasios Tsanas,et al.  Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools , 2012 .

[11]  Ke Wang,et al.  Transient stability assessment of power system using support vector machine with generator combinatorial trajectories inputs , 2013 .

[12]  Chih-Jen Lin,et al.  Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel , 2003, Neural Computation.

[13]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[15]  Wenjian Wang,et al.  Determination of the spread parameter in the Gaussian kernel for classification and regression , 2003, Neurocomputing.

[16]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[17]  Kuan-Yu Chen,et al.  Forecasting systems reliability based on support vector regression with genetic algorithms , 2007, Reliab. Eng. Syst. Saf..

[18]  Yunqian Ma,et al.  Practical selection of SVM parameters and noise estimation for SVM regression , 2004, Neural Networks.

[19]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[20]  Bijaya K. Panigrahi,et al.  A hybrid SVM-PSO model for forecasting monthly streamflow , 2013, Neural Computing and Applications.