Multiple parameter control for ant colony optimization applied to feature selection problem

The ant colony optimization algorithm (ACO) was initially developed to be a metaheuristic for combinatorial optimization problem. In scores of experiments, it is confirmed that the parameter settings in ACO have direct effects on the performance of the algorithm. However, few studies have specially reported the parameter control for ACO. The aim of this paper was to put forward some strategies to adaptively adjust the parameter in ACO and further provide a deeper understanding of ACO parameter control, including static and dynamic parameters. We choose well-known ant system (AS) and ant colony system (ACS) to be controlled by our proposed strategies. The parameters in AS and ACS include β, pheromone evaporation rate (ρ), exploration probability factor (q0) and number of ants (m). We have proposed three adaptive parameter control strategies (SI, SII and SIII) based on fuzzy logic control which adjusts ρ, q0 and m, respectively. The feature selection problem is considered for evaluating the parameter control strategies. In addition, because AS and ACS are not intrinsically fit for feature selection problem, we have modified the AS and ACS, which are named as fuzzy adaptive ant system (FAAS) and fuzzy adaptive ant colony system (FAACS), to make them more suitable for feature selection problem. Because only one parameter is allowed to be dynamically adjusted in FAAS or FAACS, the remaining parameters should be statically specified. Thus, we have developed parametric guidelines for proper combination of static parameter settings. The performance of FAAS and FAACS is compared with that of the AS-based, ACS-based, particle swarm optimization-based and genetic algorithm-based methods on a comprehensive set of 10 benchmark data sets, which are taken from UCI machine learning and StatLog databases. The numerical results and statistical analysis show that the proposed algorithms outperform significantly than other methods in terms of prediction accuracy with smaller subset of features.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[3]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Chuen-Chien Lee.  FUZZY LOGIC CONTROL SYSTEMS: FUZZY LOGIC CONTROLLER - PART I , 1990 .

[5]  Chuen-Chien Lee,et al.  Fuzzy logic in control systems: fuzzy logic controller. I , 1990, IEEE Trans. Syst. Man Cybern..

[6]  Chuen-Chien Lee,et al.  Fuzzy logic in control systems: fuzzy logic controller. II , 1990, IEEE Trans. Syst. Man Cybern..

[7]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[8]  Marco Dorigo,et al.  Optimization, Learning and Natural Algorithms , 1992 .

[9]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[10]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[11]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[13]  B. Bullnheimer,et al.  A NEW RANK BASED VERSION OF THE ANT SYSTEM: A COMPUTATIONAL STUDY , 1997 .

[14]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[15]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[18]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[19]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[20]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[21]  Angus R. Simpson,et al.  Parametric study for an ant algorithm applied to water distribution system optimization , 2005, IEEE Transactions on Evolutionary Computation.

[22]  Marco Dorigo,et al.  Ant colony optimization theory: A survey , 2005, Theor. Comput. Sci..

[23]  Jacob Zahavi,et al.  Using simulated annealing to optimize the feature selection problem in marketing applications , 2006, Eur. J. Oper. Res..

[24]  Eleonora Riva Sanseverino,et al.  Adaptive and Dynamic Ant Colony Search Algorithm for Optimal Distribution Systems Reinforcement Strategy , 2006, Applied Intelligence.

[25]  Chih-Jen Lin,et al.  Combining SVMs with Various Feature Selection Strategies , 2006, Feature Extraction.

[26]  Jun Zhang,et al.  Study of Parametric Relation in Ant Colony Optimization Approach to Traveling Salesman Problem , 2006, ICIC.

[27]  Zexuan Zhu,et al.  Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Haibin Duan,et al.  Experimental study of the adjustable parameters in basic ant colony optimization algorithm , 2007, 2007 IEEE Congress on Evolutionary Computation.

[29]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[30]  Zuren Feng,et al.  An efficient ant colony optimization approach to attribute reduction in rough set theory , 2008, Pattern Recognit. Lett..

[31]  Chuan-Wen Chiang,et al.  Ant colony optimization with parameter adaptation for multi-mode resource-constrained project scheduling , 2008, J. Intell. Fuzzy Syst..

[32]  Cheng-Lung Huang,et al.  ACO-based hybrid classification system with feature subset selection and model parameters optimization , 2009, Neurocomputing.

[33]  Wei-jie Yu,et al.  Self-adaptive ant colony system for the traveling salesman problem , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[34]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[35]  Duoqian Miao,et al.  A rough set approach to feature selection based on ant colony optimization , 2010, Pattern Recognit. Lett..

[36]  Juanying Xie,et al.  Using support vector machines with a novel hybrid feature selection method for diagnosis of erythemato-squamous diseases , 2011, Expert Syst. Appl..

[37]  Wei-Tek Tsai,et al.  Software-as-a-service (SaaS): perspectives and challenges , 2013, Science China Information Sciences.

[38]  Ying Li,et al.  An Ant Colony Optimization Based Dimension Reduction Method for High-Dimensional Datasets , 2013 .

[39]  Minghao Yin,et al.  Multiobjective Binary Biogeography Based Optimization for Feature Selection Using Gene Expression Data , 2013, IEEE Transactions on NanoBioscience.

[40]  Ping Huang,et al.  An upper (lower) bound for Max (Min) CSP , 2013, Science China Information Sciences.

[41]  Serhat Ozekes,et al.  Ant Colony Optimization Based Feature Selection Method for QEEG Data Classification , 2014, Psychiatry investigation.

[42]  Xiangtao Li,et al.  Parameter estimation for chaotic systems by hybrid differential evolution algorithm and artificial bee colony algorithm , 2014 .

[43]  Mohammad Reza Meybodi,et al.  Enriched ant colony optimization and its application in feature selection , 2014, Neurocomputing.