New Hermite orthogonal polynomial kernel and combined kernels in Support Vector Machine classifier

Support Vector Machine is a desired method for classification of different types of data, but the main obstacle to using this method is the considerable reduction of classification speed upon increase in the size of problem. In this paper, a new kernel function is proposed for SVM which is derived from Hermite orthogonal polynomials. This function improves classification accuracy as well as reduction in the number of support vectors and increases the classification speed. The overall kernel performance has been evaluated on real world data sets from UCI repository by the ten-fold cross-validation method. The combinations of Hermite function with common kernels are proposed in this paper. Experimental results reveal that the Hermite-Chebyshev kernel which is obtained from combination of Hermite and Chebyshev kernel, has the best performance in the number of support vectors and Hermite-Gussian kernel that is produced from combination Hermite and Gussian kernel, has the first rank in the error rate among all experimental kernels. On the other hand the Hermite proposed method has the least number of support vectors and best performance in the error rate in comparison with common kernels, and lowest required time among all kernels. A new kernel function has been proposed by using Hermite orthogonal polynomial.New combined kernels resulted by combining Hermite and common kernels have been proposed.The proposed methods have been compared with three common kernels on real and synthetic data sets.Hermite kernel function has the lowest required time for classification.The proposed methods yield better error rate and they have the best support vectors reduction significantly.

[1]  Ho Gi Jung,et al.  Support Vector Number Reduction: Survey and Experimental Evaluations , 2014, IEEE Transactions on Intelligent Transportation Systems.

[2]  Mo Dai,et al.  Image reconstruction from continuous Gaussian-Hermite moments implemented by discrete algorithm , 2012, Pattern Recognit..

[3]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[4]  Javad Hamidzadeh,et al.  IRDDS: Instance reduction based on Distance-based decision surface , 2015 .

[5]  Jih Pin Yeh,et al.  Optimal reduction of solutions for support vector machines , 2009, Appl. Math. Comput..

[6]  Juan Carlos Gómez,et al.  Legendre polynomials based feature extraction for online signature verification. Consistency analysis of feature combinations , 2014, Pattern Recognit..

[7]  Jason Weston,et al.  Breaking SVM Complexity with Cross-Training , 2004, NIPS.

[8]  Bor-Chen Kuo,et al.  A Kernel-Based Feature Selection Method for SVM With RBF Kernel for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[9]  Hadi Sadoghi Yazdi,et al.  IRAHC: Instance Reduction Algorithm using Hyperrectangle Clustering , 2015, Pattern Recognit..

[10]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Federico Girosi,et al.  An Equivalence Between Sparse Approximation and Support Vector Machines , 1998, Neural Computation.

[12]  Jiang-She Zhang,et al.  Reducing examples to accelerate support vector regression , 2007, Pattern Recognit. Lett..

[13]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[14]  Hakan A. Çirpan,et al.  A set of new Chebyshev kernel functions for support vector machine pattern classification , 2011, Pattern Recognit..

[15]  Chia-Hui Huang,et al.  A reduced support vector machine approach for interval regression analysis , 2012, Inf. Sci..

[16]  Xiangyang Wang,et al.  A new SVM-based relevance feedback image retrieval using probabilistic feature and weighted kernel function , 2016, J. Vis. Commun. Image Represent..

[17]  Young-Chan Lee,et al.  Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters , 2005, Expert Syst. Appl..

[18]  Kar-Ann Toh,et al.  Exploiting the relationships among several binary classifiers via data transformation , 2014, Pattern Recognit..

[19]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[20]  Wentao Mao,et al.  An adaptive support vector regression based on a new sequence of unified orthogonal polynomials , 2013, Pattern Recognit..

[21]  S. Gunn.  Support Vector Machines for Classification and Regression , 1998 .

[22]  P. N. Suganthan,et al.  A kernel-ensemble bagging support vector machine , 2012, 2012 12th International Conference on Intelligent Systems Design and Applications (ISDA).

[23]  Bing Li,et al.  State recognition of viscoelastic sandwich structures based on permutation entropy and generalized Chebyshev support vector machine , 2015, 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings.

[24]  Zhiping Lin,et al.  A center sliding Bayesian binary classifier adopting orthogonal polynomials , 2015, Pattern Recognit..

[25]  Johan A. K. Suykens,et al.  Reducing the Number of Support Vectors of SVM Classifiers Using the Smoothed Separable Case Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Edwin R. Hancock,et al.  A Graph Kernel Based on the Jensen-Shannon Representation Alignment , 2015, IJCAI.

[27]  Mo Dai,et al.  Rotation and translation invariants of Gaussian-Hermite moments , 2011, Pattern Recognit. Lett..

[28]  Lorenzo Bruzzone,et al.  Semisupervised Kernel Feature Extraction for Remote Sensing Image Analysis , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[30]  Ingo Steinwart,et al.  Sparseness of Support Vector Machines---Some Asymptotically Sharp Bounds , 2003, NIPS.

[31]  Yixin Chen,et al.  A sparse support vector machine approach to region-based image categorization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Lijuan Cao,et al.  A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine , 2003, Neurocomputing.

[33]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[34]  Jun Zhou,et al.  Object Detection Via Structural Feature Selection and Shape Model , 2013, IEEE Transactions on Image Processing.

[35]  Marc G. Genton,et al.  Classes of Kernels for Machine Learning: A Statistics Perspective , 2002, J. Mach. Learn. Res..

[36]  Rene F. Swarttouw,et al.  Orthogonal polynomials , 2020, NIST Handbook of Mathematical Functions.

[37]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .