On global, local, mixed and neighborhood kernels for support vector machines

Abstract The basic principles of the support vector machine (SVM) are analyzed. Two approaches to constructing a kernel function which takes into account some local properties of a problem are considered. The first one deals with interactions between neighboring pixels in an image and the second with proximity of the objects in the input space. In the former case, this is equivalent to feature selection and the efficiency of this approach is demonstrated by an application to Texture Recognition. In the latter case, this approach may be considered as either a kind of local algorithm or as a mixture of local and global ones. We demonstrate that the use of such kernels increases the domain of SVM applications.

[1]  Bernhard Schölkopf,et al.  Incorporating Invariances in Support Vector Learning Machines , 1996, ICANN.

[2]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[3]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[4]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[5]  Ralf Der,et al.  Building Nonlinear Data Models with Self-Organizing Maps , 1996, ICANN.

[6]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[7]  Anil K. Jain,et al.  Random field models in image analysis , 1989 .

[8]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[9]  Victor L. Brailovsky,et al.  On domain knowledge and feature selection using a support vector machine , 1999, Pattern Recognit. Lett..

[10]  Bernhard Schölkopf,et al.  Prior Knowledge in Support Vector Kernels , 1997, NIPS.

[11]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[14]  Bernhard Schölkopf,et al.  Support Vector methods in learning and feature extraction , 1998 .

[15]  B. Schölkopf,et al.  Advances in kernel methods: support vector learning , 1999 .