A New Composition Method of Admissible Support Vector Kernel Based on Reproducing Kernel

Kernel function, which allows the formulation of nonlinear variants of any algorithm that can be cast in terms of dot products, makes the Support Vector Machines (SVM) have been successfully applied in many fields, e.g. classification and regression. The importance of kernel has motivated many studies on its composition. It’s well-known that reproducing kernel (R.K) is a useful kernel function which possesses many properties, e.g. positive definiteness, reproducing property and composing complex R.K by simple operation. There are two popular ways to compute the R.K with explicit form. One is to construct and solve a specific differential equation with boundary value whose handicap is incapable of obtaining a unified form of R.K. The other is using a piecewise integral of the Green function associated with a differential operator L. The latter benefits the computation of a R.K with a unified explicit form and theoretical analysis, whereas there are relatively later studies and fewer practical computations. In this paper, a new algorithm for computing a R.K is presented. It can obtain the unified explicit form of R.K in general reproducing kernel Hilbert space. It avoids constructing and solving the complex differential equations manually and benefits an automatic, flexible and rigorous computation for more general RKHS. In order to validate that the R.K computed by the algorithm can be used in SVM well, some illustrative examples and a comparison between R.K and Gaussian kernel (RBF) in support vector regression are presented. The result shows that the performance of R.K is close or slightly superior to that of RBF. Keywords—admissible support vector kernel, reproducing kernel, reproducing kernel Hilbert space, Green function, support vector regression

[1]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[2]  Jun Wang,et al.  A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension , 2004, IEEE Transactions on Knowledge and Data Engineering.

[3]  D. Anguita,et al.  The effect of quantization on support vector machines with Gaussian kernel , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[4]  Alexander J. Smola,et al.  Regularization with Dot-Product Kernels , 2000, NIPS.

[5]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[6]  Bernhard Schölkopf,et al.  The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[7]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Bernhard Schölkopf,et al.  Comparing support vector machines with Gaussian kernels to radial basis function classifiers , 1997, IEEE Trans. Signal Process..

[10]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[11]  Limin Ma,et al.  Kernel based approximation in Sobolev spaces with radial basis functions , 2009, Appl. Math. Comput..

[12]  Victor-Emil Neagoe,et al.  Inversion of the Van der Monde matrix , 1996, IEEE Signal Processing Letters.

[13]  Liu Yuncai.  Performance Analysis of Support Vector Machines with Gauss Kernel , 2003 .

[14]  Roland Opfer,et al.  Multiscale kernels , 2006, Adv. Comput. Math..

[15]  Bernhard Schölkopf,et al.  Face Detection - Efficient and Rank Deficient , 2004, NIPS.

[16]  Mohamed Mohandes,et al.  Support vector machines for wind speed prediction , 2004 .

[17]  B. Schölkopf,et al.  General cost functions for support vector regression. , 1998 .

[18]  Zhizhong Wang,et al.  Model optimizing and feature selecting for support vector regression in time series forecasting , 2008, Neurocomputing.

[19]  Jing-Xu Liu,et al.  An empirical assessment on the robustness of support vector regression with different kernels , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[20]  Zhang Qin-li,et al.  A New Method for Computing Reproducing Kernels , 2002 .

[21]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[22]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Stefan Bergman.  The approximation of functions satisfying a linear partial differential equation , 1940 .

[25]  R. Schaback.  A unified theory of radial basis functions Native Hilbert spaces for radial basis functions II , 2000 .

[26]  Bernhard Schölkopf,et al.  Comparison of View-Based Object Recognition Algorithms Using Realistic 3D Models , 1996, ICANN.

[27]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[28]  R. V. Churchill,et al.  Lectures on Fourier Integrals , 1959 .

[29]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[30]  Ming-gen Cui,et al.  Two-dimensional reproducing kernel and surface interpolation , 1986 .

[31]  Holger Wendland,et al.  Approximation by positive definite kernels , 2002 .

[32]  Yuesheng Li.  On the recurrence relations for B-splines defined by certain L-splines , 1985 .

[33]  Junbin Gao,et al.  Mean field method for the support vector machine regression , 2003, Neurocomputing.

[34]  Colin Fyfe,et al.  The kernel self-organising map , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[35]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[36]  Ping Zhu,et al.  Metamodel-based lightweight design of B-pillar with TWB structure via support vector regression , 2010 .

[37]  Minggen Cui,et al.  Nonlinear Numerical Analysis in Reproducing Kernel Space , 2009 .

[38]  Gérard Bloch,et al.  Support vector regression from simulation data and few experimental samples , 2008, Inf. Sci..

[39]  G. Wahba.  Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV , 1999 .

[40]  Bernhard Schölkopf,et al.  Improving the Accuracy and Speed of Support Vector Machines , 1996, NIPS.

[41]  Junbin Gao,et al.  On a Class of Support Vector Kernels Based on Frames in Function Hilbert Spaces , 2001, Neural Computation.