Model Selection for Support Vector Classifiers via Genetic Algorithms. An Application to Medical Decision Support

This paper addresses the problem of tuning hyperparameters in support vector machine modeling. A Genetic Algorithm-based wrapper, which seeks to evolve hyperparameter values using an empirical error estimate as a fitness function, is proposed and experimentally evaluated on a medical dataset. Model selection is then fully automated. Unlike other hyperparameters tuning techniques, genetic algorithms do not require supplementary information making them well suited for practical purposes. This approach was motivated by an application where the number of parameters to adjust is greater than one. This method produces satisfactory results.

[1]  S. Sathiya Keerthi,et al.  Evaluation of simple performance measures for tuning SVM hyperparameters , 2003, Neurocomputing.

[2]  S. Sathiya Keerthi,et al.  Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms , 2002, IEEE Trans. Neural Networks.

[3]  N. Lavrac,et al.  Intelligent Data Analysis in Medicine and Pharmacology , 1997 .

[4]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[5]  John Shawe-Taylor,et al.  Optimizing Classifers for Imbalanced Training Sets , 1998, NIPS.

[6]  Ching Y. Suen,et al.  Empirical error based optimization of SVM kernels: application to digit image recognition , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[7]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[8]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[11]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[12]  P. Francioli,et al.  Nosocomial infections in Swiss university hospitals: a multi-centre survey and review of the published experience. Swiss-Noso Network. , 1999, Schweizerische medizinische Wochenschrift.

[13]  Thorsten Joachims,et al.  Estimating the Generalization Performance of an SVM Efficiently , 2000, ICML.

[14]  R. Fletcher Practical Methods of Optimization , 1988 .

[15]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .