A comparative study of suboptimal branch and bound algorithms

Abstract The branch and bound algorithm is widely known as an efficient approach for selecting optimal feature subsets. If the optimality of the solution is allowed to be compromised, it is possible to further improve the search speed of the branch and bound algorithm. This paper studies the look-ahead search strategy which can eliminate many solutions deemed to be suboptimal early in the branch and bound search. We propose ways to incorporate the look-ahead search scheme into four major branch and bound algorithms, namely the basic branch and bound algorithm, the ordered branch and bound algorithm, the fast branch and bound algorithm, and the adaptive branch and bound algorithm. A comparative study of the look-ahead scheme in terms of the computational cost and the solution quality on these suboptimal branch and bound algorithms is carried out on real data sets. Furthermore, we test the feasible use of suboptimal branch and bound algorithms on a high-dimensional data set and compare its performance with other well-known suboptimal feature selection algorithms.

[1]  Minqiang Li,et al.  A hybrid coevolutionary algorithm for designing fuzzy classifiers , 2009, Inf. Sci..

[2]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[3]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[4]  Marco Dorigo,et al.  Ant colony optimization theory: A survey , 2005, Theor. Comput. Sci..

[5]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[6]  Ed Keedwell,et al.  Ant colony optimisation to identify genetic variant association with type 2 diabetes , 2011, Inf. Sci..

[7]  Mao Ye,et al.  A tabu search approach for the minimum sum-of-squares clustering problem , 2008, Inf. Sci..

[8]  Sanghyun Park,et al.  Direct integration of microarrays for selecting informative genes and phenotype classification , 2008, Inf. Sci..

[9]  David Casasent,et al.  Adaptive branch and bound algorithm for selecting optimal features , 2007, Pattern Recognit. Lett..

[10]  Maghsud Solimanpur,et al.  Optimal solution for the two-dimensional facility layout problem using a branch-and-bound algorithm , 2008, Comput. Ind. Eng..

[11]  Chengbin Chu,et al.  A branch and bound algorithm to minimize total weighted completion time on identical parallel machines with job release dates , 2006, 2006 International Conference on Service Systems and Service Management.

[12]  Jitamitra Desai,et al.  A global optimization algorithm for reliable network design , 2010, Eur. J. Oper. Res..

[13]  David Casasent,et al.  An improvement on floating search algorithms for feature subset selection , 2009, Pattern Recognit..

[14]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[15]  Lorenzo Bruzzone,et al.  A new search algorithm for feature selection in hyperspectral remote sensing images , 2001, IEEE Trans. Geosci. Remote. Sens..

[16]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[17]  Bo Yang,et al.  Data gravitation based classification , 2009, Inf. Sci..

[18]  Josef Kittler,et al.  Fast branch & bound algorithms for optimal feature selection , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[20]  Francesc J. Ferri,et al.  Comparative study of techniques for large-scale feature selection* *This work was suported by a SERC grant GR/E 97549. The first author was also supported by a FPI grant from the Spanish MEC, PF92 73546684 , 1994 .

[21]  Richard Weber,et al.  A wrapper method for feature selection using Support Vector Machines , 2009, Inf. Sci..

[22]  Baozong Yuan,et al.  A more efficient branch and bound algorithm for feature selection , 1993, Pattern Recognit..

[23]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[24]  Songyot Nakariyakul,et al.  On the suboptimal solutions using the adaptive branch and bound algorithm for feature selection , 2008, 2008 International Conference on Wavelet Analysis and Pattern Recognition.

[25]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognit. Lett..

[26]  Pavel Paclík,et al.  Adaptive floating search methods in feature selection , 1999, Pattern Recognit. Lett..