The Support Feature Machine for Classifying with the Least Number of Features

We propose the so-called Support Feature Machine (SFM) as a novel approach to feature selection for classification, based on minimisation of the zero norm of a separating hyperplane. Thus, a classifier with inherent feature selection capabilities is obtained within a single training run. Results on toy examples demonstrate that this method is able to identify relevant features very effectively.

[1]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[2]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[3]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[4]  Bernhard Schölkopf,et al.  Use of the Zero-Norm with Linear Models and Kernel Methods , 2003, J. Mach. Learn. Res..

[5]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[6]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[7]  Jinbo Bi,et al.  Dimensionality Reduction via Sparse Support Vector Machines , 2003, J. Mach. Learn. Res..