Choosing Multiple Parameters for Support Vector Machines

The problem of automatically tuning multiple parameters for pattern recognition Support Vector Machines (SVMs) is considered. This is done by minimizing some estimates of the generalization error of SVMs using a gradient descent algorithm over the set of parameters. Usual methods for choosing parameters, based on exhaustive search become intractable as soon as the number of parameters exceeds two. Some experimental results assess the feasibility of our approach for a large number of parameters (more than 100) and demonstrate an improvement of generalization performance.

[1]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[2]  Lars Kai Hansen,et al.  Adaptive Regularization in Neural Network Modeling , 1996, Neural Networks: Tricks of the Trade.

[3]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[4]  Nello Cristianini,et al.  Dynamically Adapting Kernels in Support Vector Machines , 1998, NIPS.

[5]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[6]  P. Maher,et al.  Handbook of Matrices , 1999, The Mathematical Gazette.

[7]  David Haussler,et al.  Probabilistic kernel regression models , 1999, AISTATS.

[8]  Olivier Chapelle,et al.  Model Selection for Support Vector Machines , 1999, NIPS.

[9]  Thomas Serre,et al.  Feature Selection for Face Detection , 2000 .

[10]  V. Vapnik,et al.  Bounds on Error Expectation for Support Vector Machines , 2000, Neural Computation.

[11]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[12]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[13]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[14]  Massimiliano Pontil,et al.  Face Detection in Still Gray Images , 2000 .

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  J. Frédéric Bonnans,et al.  Perturbation Analysis of Optimization Problems , 2000, Springer Series in Operations Research.

[17]  Thorsten Joachims,et al.  Estimating the Generalization Performance of an SVM Efficiently , 2000, ICML.

[18]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[19]  P. Bartlett,et al.  Gaussian Processes and SVM: Mean Field and Leave-One-Out , 2000 .

[20]  Hansong Zhang,et al.  Gacv for support vector machines , 2000 .

[21]  Yoshua Bengio,et al.  Gradient-Based Optimization of Hyperparameters , 2000, Neural Computation.

[22]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[23]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.