Machine learning using hyperkernels

We expand on the problem of learning a kernel via a RKHS on the space of kernels itself. The resulting optimization problem is shown to have a semidefinite programming solution. We demonstrate that it is possible to learn the kernel for various formulations of machine learning problems. Specifically, we provide mathematical programming formulations and experimental results for the C-SVM, ν-SVM and Lagrangian SVM for classification on UCI data, and novelty detection.

[1]  A. Albert.  Conditions for Positive and Nonnegative Definiteness in Terms of Pseudoinverses , 1969 .

[2]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[3]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[4]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[5]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[6]  P. Bartlett,et al.  Gaussian Processes and SVM: Mean Field and Leave-One-Out , 2000 .

[7]  Katya Scheinberg,et al.  Efficient SVM Training Using Low-Rank Kernel Representations , 2002, J. Mach. Learn. Res..

[8]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[9]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[10]  Tong Zhang,et al.  Some Sparse Approximation Bounds for Regression Problems , 2001, International Conference on Machine Learning.

[11]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[12]  Koby Crammer,et al.  Kernel Design Using Boosting , 2002, NIPS.

[13]  Olivier Bousquet,et al.  On the Complexity of Learning the Kernel Matrix , 2002, NIPS.

[14]  Kristin P. Bennett,et al.  A Pattern Search Method for Model Selection of Support Vector Regression , 2002, SDM.

[15]  Alexander J. Smola,et al.  Hyperkernels , 2002, NIPS.

[16]  Kurt Hornik,et al.  The support vector machine under test , 2003, Neurocomputing.

[17]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[18]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[19]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.