Low-cost model selection for SVMs using local features

Many practical engineering applications require the usage of accurate automatic decision systems, usually operating under tight computational constraints. Support Vector Machines (SVMs) endowed with a Radial Basis Function (RBF) as kernel are broadly accepted as the current state of the art for decision problems, but require cross-validation to select the free parameters, which is computationally costly. In this work we investigate low-cost methods to select the spread parameter in SVMs with an RBF kernel. Our proposal relies on the use of simple local methods that gather information about the local structure of each dataset. Empirical results in UCI datasets show that the proposed methods can be used as a fast alternative to the standard cross-validation procedure, with the additional advantage of avoiding the (often heuristic) task of a priori fixing the values of the spread parameter to be explored.

[1]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[2]  Sancho Salcedo-Sanz,et al.  Improving the training time of support vector regression algorithms through novel hyper-parameters search space reductions , 2009, Neurocomputing.

[3]  Jason Weston,et al.  Breaking SVM Complexity with Cross-Training , 2004, NIPS.

[4]  K. R. Al-Balushi,et al.  Artificial neural networks and support vector machines with genetic algorithm for bearing fault detection , 2003 .

[5]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[6]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Fernando Díaz,et al.  A model for parameter setting based on Bayesian networks , 2008, Eng. Appl. Artif. Intell..

[9]  Christian Igel,et al.  Evolutionary tuning of multiple SVM parameters , 2005, ESANN.

[10]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[11]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[12]  S. J. Cheng,et al.  Application and comparison of computational intelligence techniques for optimal location and parameter setting of UPFC , 2010, Eng. Appl. Artif. Intell..

[13]  Ron Kohavi,et al.  Automatic Parameter Selection by Minimizing Estimated Error , 1995, ICML.

[14]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[15]  B. K. Panigrahi,et al.  ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE , 2010 .

[16]  John Shawe-Taylor,et al.  The Set Covering Machine , 2003, J. Mach. Learn. Res..

[17]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[18]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[19]  Yoshua Bengio,et al.  Gradient-Based Optimization of Hyperparameters , 2000, Neural Computation.

[20]  Kristin P. Bennett,et al.  A Pattern Search Method for Model Selection of Support Vector Regression , 2002, SDM.

[21]  Ramón Quiza Sardiñas,et al.  Genetic algorithm-based multi-objective optimization of cutting parameters in turning processes , 2006, Eng. Appl. Artif. Intell..