Feature Selection Using Combine of Genetic Algorithm and Ant Colony Optimization

Feature selection has recently been the subject of intensive research in data mining, especially for datasets with a large number of attributes. Recent work has shown that feature selection can have a positive affect on the performance of machine learning algorithms. The success of many learning algorithms in their attempts to construct models of data, hinges on the reliable identification of a small set of highly predictive attributes. The inclusion of irrelevant, redundant and noisy attributes in the model building process phase can result in poor predictive performance and increased computation. In this paper, a novel feature search procedure that utilizes combining of the Ant Colony Optimization (ACO) and genetic algorithm (GA) is presented. The ACO is a meta-heuristic inspired by the behavior of real ants in their search for the shortest paths to food sources. It looks for optimal solutions by considering both local heuristics and previous knowledge. Genetic algorithm selects the best parameters for ant colony optimization in each step. When this algorithm applied to two different classification problems, the proposed algorithm achieved very promising results.

[1]  Ron Kohavi,et al.  The Wrapper Approach , 1998 .

[2]  J. Kittler,et al.  Feature Set Search Alborithms , 1978 .

[3]  Pat Langley,et al.  Scaling to domains with irrelevant features , 1997, COLT 1997.

[4]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..

[5]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[6]  David B. Skalak,et al.  Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing Algorithms , 1994, ICML.

[7]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[8]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[9]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Feature Subset Selection , 1977, IEEE Transactions on Computers.

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  Kan Deng,et al.  On the Greediness of Feature Selection Algorithms , 1999 .

[12]  Russell Greiner,et al.  Computational learning theory and natural learning systems , 1997 .

[13]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[14]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..