Hidden space support vector machines

Hidden space support vector machines (HSSVMs) are presented in this paper. The input patterns are mapped into a high-dimensional hidden space by a set of hidden nonlinear functions and then the structural risk is introduced into the hidden space to construct HSSVMs. Moreover, the conditions for the nonlinear kernel function in HSSVMs are more relaxed, and even differentiability is not required. Compared with support vector machines (SVMs), HSSVMs can adopt more kinds of kernel functions because the positive definite property of the kernel function is not a necessary condition. The performance of HSSVMs for pattern recognition and regression estimation is also analyzed. Experiments on artificial and real-world domains confirm the feasibility and the validity of our algorithms.

[1]  L. Jones.  A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[2]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[3]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[4]  Alexander J. Smola,et al.  Regression estimation with support vector learning machines , 1996 .

[5]  G. Wahba.  Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV , 1999 .

[6]  Johan A. K. Suykens,et al.  Training multilayer perceptron classifiers based on a modified support vector method , 1999, IEEE Trans. Neural Networks.

[7]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[8]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[9]  Saburou Saitoh,et al.  Theory of Reproducing Kernels and Its Applications , 1988 .

[10]  Robert O. Winder,et al.  Enumeration of Seven-Argument Threshold Functions , 1965, IEEE Trans. Electron. Comput..

[11]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[12]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[13]  Alexander J. Smola,et al.  Support Vector Machine Reference Manual , 1998 .

[14]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[15]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[16]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[17]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[18]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[19]  H. N. Mhaskar,et al.  Neural Networks for Optimal Approximation of Smooth and Analytic Functions , 1996, Neural Computation.

[20]  Fa-Long Luo,et al.  Applied neural networks for signal processing , 1997 .

[21]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[22]  Bernhard Schölkopf,et al.  Extracting Support Data for a Given Task , 1995, KDD.

[23]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[24]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .

[25]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[26]  K. Lang,et al.  Learning to tell two spirals apart , 1988 .

[27]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[28]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[29]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.