Adaptive branch and bound algorithm for selecting optimal features

We propose a new adaptive branch and bound algorithm for selecting the optimal subset of features in pattern recognition applications. The algorithm improves the search speed by avoiding unnecessary criterion function calculations at nodes in the solution tree. Our algorithm includes the following new properties: (i) ordering the tree nodes by the significance of features during construction of the tree, (ii) obtaining a large ''good'' initial bound by a floating search method, (iii) a new method to select an initial starting search level in the tree, and (iv) a new adaptive jump search strategy to select subsequent search levels to avoid redundant criterion function calculations. Our experimental results for four different databases demonstrate that our method is significantly faster than other versions of the branch and bound algorithm when the database has more than 30 features.

[1]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[2]  David Casasent,et al.  Waveband selection for hyperspectral data: optimal feature selection , 2003, SPIE Defense + Commercial Sensing.

[3]  Xue-wen Chen.  An improved branch and bound algorithm for feature selection , 2003, Pattern Recognit. Lett..

[4]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[5]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[6]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[7]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[8]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[9]  Josef Kittler,et al.  Fast branch & bound algorithms for optimal feature selection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[11]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Francesc J. Ferri,et al.  Comparative study of techniques for large-scale feature selection* *This work was suported by a SERC grant GR/E 97549. The first author was also supported by a FPI grant from the Spanish MEC, PF92 73546684 , 1994 .

[13]  Baozong Yuan,et al.  A more efficient branch and bound algorithm for feature selection , 1993, Pattern Recognit..

[14]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[15]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .