Hybrid Genetic Algorithms for Feature Selection

This paper proposes a novel hybrid genetic algorithm for feature selection. Local search operations are devised and embedded in hybrid GAs to fine-tune the search. The operations are parameterized in terms of their fine-tuning power, and their effectiveness and timing requirements are analyzed and compared. The hybridization technique produces two desirable effects: a significant improvement in the final performance and the acquisition of subset-size control. The hybrid GAs showed better convergence properties compared to the classical GAs. A method of performing rigorous timing analysis was developed, in order to compare the timing requirement of the conventional and the proposed algorithms. Experiments performed with various standard data sets revealed that the proposed hybrid GA is superior to both a simple GA and sequential search algorithms.

[1]  Byung Ro Moon,et al.  Local search-embedded genetic algorithms for feature selection , 2002, Object recognition supported by user interaction for service robots.

[2]  P. Langley.  Selection of Relevant Features in Machine Learning , 1994 .

[3]  Ching Y. Suen,et al.  Analysis of Class Separation and Combination of Class-Dependent Features for Handwriting Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Byung Ro Moon,et al.  Genetic Algorithm and Graph Partitioning , 1996, IEEE Trans. Computers.

[5]  Josef Kittler,et al.  Using feature selection to aid an iconic search through an image database , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Ching Y. Suen,et al.  Distance features for neural network-based recognition of handwritten characters , 1998, International Journal on Document Analysis and Recognition.

[7]  Lakhmi C. Jain,et al.  Nearest neighbor classifier: Simultaneous editing and feature selection , 1999, Pattern Recognit. Lett..

[8]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognition Letters.

[9]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[10]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[11]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[12]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[13]  Donald E. Brown,et al.  Fast generic selection of features for neural network classifiers , 1992, IEEE Trans. Neural Networks.

[14]  M.J. Martin-Bautista,et al.  A survey of genetic feature selection in mining issues , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[15]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[17]  Selwyn Piramuthu,et al.  Artificial Intelligence and Information Technology Evaluating feature selection methods for learning in data mining applications , 2004 .

[18]  Michael S. Lew,et al.  Principles of Visual Information Retrieval , 2001, Advances in Pattern Recognition.

[19]  Alberto Del Bimbo,et al.  Visual information retrieval , 1999 .

[20]  B. Julstrom,et al.  Design of vector quantization codebooks using a genetic algorithm , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[21]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[22]  Dirk Van Gucht,et al.  The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem , 1989 .

[23]  Mineichi Kudo,et al.  Comparison of algorithms that select features for pattern classifiers , 2000, Pattern Recognit..

[24]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[25]  Jack Sklansky,et al.  On Automatic Feature Selection , 1988, Int. J. Pattern Recognit. Artif. Intell..

[26]  Yanxi Liu,et al.  A classification based similarity metric for 3D image retrieval , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[27]  Hiroshi Ishikawa.  Multi-scale feature selection in stereo , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[28]  Baozong Yuan,et al.  A more efficient branch and bound algorithm for feature selection , 1993, Pattern Recognit..

[29]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[30]  Alexey Tsymbal,et al.  Advanced local feature selection in medical diagnostics , 2000, Proceedings 13th IEEE Symposium on Computer-Based Medical Systems. CBMS 2000.

[31]  Chih-Cheng Hung,et al.  A comparative study of remotely sensed data classification using principal components analysis and divergence , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[32]  A. Meyer-Bäse.  Feature Selection and Extraction , 2004 .