A wrapper method for feature selection using Support Vector Machines

We introduce a novel wrapper Algorithm for Feature Selection, using Support Vector Machines with kernel functions. Our method is based on a sequential backward selection, using the number of errors in a validation subset as the measure to decide which feature to remove in each iteration. We compare our approach with other algorithms like a filter method or Recursive Feature Elimination SVM to demonstrate its effectiveness and efficiency.

[1]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[2]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[3]  Özge Uncu,et al.  A novel feature selection approach: Combining feature wrappers and filters , 2007, Inf. Sci..

[4]  Wei-Min Shen,et al.  Data Preprocessing and Intelligent Data Analysis , 1997, Intell. Data Anal..

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[6]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[7]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[8]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[9]  Alain Rakotomamonjy,et al.  Variable Selection Using SVM-based Criteria , 2003, J. Mach. Learn. Res..

[10]  Chih-Chieh Yang,et al.  Multiclass SVM-RFE for product form feature selection , 2008, Expert Syst. Appl..

[11]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[12]  Bernhard Schölkopf,et al.  Use of the Zero-Norm with Linear Models and Kernel Methods , 2003, J. Mach. Learn. Res..

[13]  Michalis E. Blazadonakis,et al.  Wrapper filtering criteria via linear neuron and kernel approaches , 2008, Comput. Biol. Medicine.

[14]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[15]  Richard Weber,et al.  Linear Penalization Support Vector Machines for Feature Selection , 2005, PReMI.

[16]  Yi Liu,et al.  FS_SFS: A novel feature selection method for support vector machines , 2006, Pattern Recognit..

[17]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[18]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[19]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[20]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[21]  Masoud Nikravesh,et al.  Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing) , 2006 .

[22]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[23]  Jude W. Shavlik,et al.  Machine Learning: Proceedings of the Fifteenth International Conference , 1998 .

[24]  Jason Weston,et al.  Embedded Methods , 2006, Feature Extraction.