An advanced ACO algorithm for feature subset selection

Abstract Feature selection is an important task for data analysis and information retrieval processing, pattern classification systems, and data mining applications. It reduces the number of features by removing noisy, irrelevant and redundant data. In this paper, a novel feature selection algorithm based on Ant Colony Optimization (ACO), called Advanced Binary ACO (ABACO), is presented. Features are treated as graph nodes to construct a graph model and are fully connected to each other. In this graph, each node has two sub-nodes, one for selecting and the other for deselecting the feature. Ant colony algorithm is used to select nodes while ants should visit all features. The use of several statistical measures is examined as the heuristic function for visibility of the edges in the graph. At the end of a tour, each ant has a binary vector with the same length as the number of features, where 1 implies selecting and 0 implies deselecting the corresponding feature. The performance of proposed algorithm is compared to the performance of Binary Genetic Algorithm (BGA), Binary Particle Swarm Optimization (BPSO), CatfishBPSO, Improved Binary Gravitational Search Algorithm (IBGSA), and some prominent ACO-based algorithms on the task of feature selection on 12 well-known UCI datasets. Simulation results verify that the algorithm provides a suitable feature subset with good classification accuracy using a smaller feature set than competing feature selection methods.

[1]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[2]  K. R. Chandran,et al.  An enhanced ACO algorithm to select features for text categorization and its parallelization , 2012, Expert Syst. Appl..

[3]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[4]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[5]  Sheng Ding,et al.  Feature Selection Based F-Score and ACO Algorithm in Support Vector Machine , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.

[6]  Gian Luca Foresti,et al.  A distributed probabilistic system for adaptive regulation of image processing parameters , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[8]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[9]  Lei Liu,et al.  Feature selection with dynamic mutual information , 2009, Pattern Recognit..

[10]  Yixin Chen,et al.  Efficient ant colony optimization for image feature selection , 2013, Signal Process..

[11]  Kazuyuki Murase,et al.  A new wrapper feature selection approach using neural network , 2010, Neurocomputing.

[12]  Lale Özbakir,et al.  TACO-miner: An ant colony based algorithm for rule extraction from trained neural networks , 2009, Expert Syst. Appl..

[13]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[14]  Nasser Ghasem-Aghaee,et al.  Using Ant Colony Optimization-Based Selected Features for Predicting Post-synaptic Activity in Proteins , 2008, EvoBIO.

[15]  Jose Miguel Puerta,et al.  A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets , 2011, Pattern Recognit. Lett..

[16]  Hossein Nezamabadi-pour,et al.  Feature subset selection using improved binary gravitational search algorithm , 2014, J. Intell. Fuzzy Syst..

[17]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Brian J. d'Auriol,et al.  A novel feature selection method based on normalized mutual information , 2011, Applied Intelligence.

[19]  Thomas A. Runkler,et al.  Two cooperative ant colonies for feature selection using fuzzy models , 2010, Expert Syst. Appl..

[20]  Nasser Ghasem-Aghaee,et al.  Application of ant colony optimization for feature selection in text categorization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[21]  Hong Hu,et al.  Feature selection using the hybrid of ant colony optimization and mutual information for the forecaster , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[22]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..

[23]  Hossein Nezamabadi-pour,et al.  Facing the classification of binary problems with a GSA-SVM hybrid system , 2013, Math. Comput. Model..

[24]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[25]  Ahmed Al-Ani,et al.  Feature Subset Selection Using Ant Colony Optimization , 2008 .

[26]  Giles M. Foody,et al.  Feature Selection for Classification of Hyperspectral Data by SVM , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Chao-Ton Su,et al.  Applying electromagnetism-like mechanism for feature selection , 2011, Inf. Sci..

[28]  Zohar Nussinov,et al.  A Replica Inference Approach to Unsupervised Multi-Scale Image Segmentation , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  Richard Jensen,et al.  Combining rough and fuzzy sets for feature selection , 2004 .

[30]  Duoqian Miao,et al.  A rough set approach to feature selection based on ant colony optimization , 2010, Pattern Recognit. Lett..

[31]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Nasser Ghasem-Aghaee,et al.  A novel ACO-GA hybrid algorithm for feature selection in protein function prediction , 2009, Expert Syst. Appl..

[33]  Hossein Nezamabadi-pour,et al.  A simultaneous feature adaptation and feature selection method for content-based image retrieval systems , 2013, Knowl. Based Syst..

[34]  Ron Kohavi,et al.  Wrappers for feature selection , 1997 .

[35]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[36]  Corso Elvezia,et al.  Ant colonies for the traveling salesman problem , 1997 .

[37]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[38]  Huang Lin,et al.  Hybrid feature selection algorithm based on dynamic weighted ant colony algorithm , 2010, 2010 International Conference on Machine Learning and Cybernetics.

[39]  Li-Yeh Chuang,et al.  Chaotic maps based on binary particle swarm optimization for feature selection , 2011, Appl. Soft Comput..

[40]  Kazuyuki Murase,et al.  A new local search based hybrid genetic algorithm for feature selection , 2011, Neurocomputing.

[41]  Christian Blum,et al.  Ant colony optimization: Introduction and recent trends , 2005 .

[42]  LeeSungyoung,et al.  A novel feature selection method based on normalized mutual information , 2012 .

[43]  M Dorigo,et al.  Ant colonies for the travelling salesman problem. , 1997, Bio Systems.

[44]  Sreeram Ramakrishnan,et al.  A hybrid approach for feature subset selection using neural networks and ant colony optimization , 2007, Expert Syst. Appl..

[45]  Xiaoming Xu,et al.  A hybrid genetic algorithm for feature selection wrapper based on mutual information , 2007, Pattern Recognit. Lett..

[46]  Cheng-Lung Huang,et al.  ACO-based hybrid classification system with feature subset selection and model parameters optimization , 2009, Neurocomputing.

[47]  Mahesh Pal,et al.  Hybrid genetic algorithm for feature selection with hyperspectral data , 2013 .

[48]  Hossein Nezamabadi-pour,et al.  A new feature selection algorithm based on binary ant colony optimization , 2013, The 5th Conference on Information and Knowledge Technology.

[49]  Dervis Karaboga,et al.  Designing digital IIR filters using ant colony optimisation algorithm , 2004, Eng. Appl. Artif. Intell..