Estimating Parameters of Kernel Functions in Support Vector Learning

The selection and modification of kernel functions is a very important but rarely studied problem in the field of support vector learning. However, the kernel function of a support vector machine has great influence on its performance. The kernel function projects the dataset from the original data space into the feature space, and therefore the problems which can't be done in low dimensions could be done in a higher dimension through the transform of the kernel function. In this paper, we adopt the FCM clustering algorithm to group data patterns into clusters, and then use a statistical approach to calculate the standard deviation of each pattern with respect to the other patterns in the same cluster. Therefore we can make a proper estimation on the distribution of kernel functions. Experimental results have shown that our approach can derive better kernel functions than other methods, and also can have better learning and generalization abilities.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  R. Nakano,et al.  Yet faster method to optimize SVR hyperparameters based on minimizing cross-validation error , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[3]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  Ryohei Nakano,et al.  Optimizing Support Vector regression hyperparameters based on cross-validation , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[5]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[6]  Chih-Jen Lin,et al.  Training v-Support Vector Regression: Theory and Algorithms , 2002, Neural Computation.

[7]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[10]  Chen-Chia Chuang,et al.  A novel approach for the hyperparameters of support vector regression , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[12]  Jun Wang,et al.  A support vector machine with a hybrid kernel and minimal Vapnik-Chervonenkis dimension , 2004, IEEE Transactions on Knowledge and Data Engineering.

[13]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[14]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..