Graph structure optimization of Genetic Network Programming with ant colony mechanism in deterministic and stochastic environments

Abstract Evolutionary Algorithms are of the most successful algorithms in solving various optimization problems. Genetic network programming is one of the Evolutionary Algorithms with good capabilities in agent control problems. In this algorithm, the individuals’ structure is a directed graph. Using this structure, it is possible to model the solution of many complex problems. However, in this algorithm, crossover and mutation operators repeatedly break the structures of individuals and make new ones. Although this can lead to better structures, it may break suitable structures in elite individuals. Meanwhile, in stochastic environments, each time an individual is evaluated, it leads to different fitness values. So, calculating the fitness value of individuals requires evaluating each individual several times. This extremely decreases the evolution process speed. In this paper, inspired by mechanisms of ant colony algorithm, a new method is proposed to prevent the algorithm from iteratively breaking down the structures of individuals. This method improves the performance of individuals from one generation to the next using a constructive process. Unlike generative process that the individuals are generated by combination of some others, in constructive process they are produced according to the experience of previous generations. Using this mechanism, we not only prevent breaking suitable structures but also can manage uncertainty in stochastic environments. Our proposed method is used to solve two agent control problems when the environment is deterministic or stochastic. The results show that the proposed algorithm has very high ability in creating an efficient decision making strategies especially in stochastic environments.

[1]  Edward J. Williams,et al.  A Survey on Ant Colony Optimization, Particle Swarm Optimization, and Cuckoo Algorithms , 2018 .

[2]  Shingo Mabu,et al.  Genetic Network Programming with Reinforcement Learning and Its Performance Evaluation , 2004, GECCO.

[3]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[4]  Shingo Mabu,et al.  Towards the Maintenance of Population Diversity: A Hybrid Probabilistic Model Building Genetic Network Programming , 2010 .

[5]  Tim Hendtlass,et al.  Improving exploration in Ant Colony Optimisation with antennation , 2012, 2012 IEEE Congress on Evolutionary Computation.

[6]  Shingo Mabu,et al.  An extended probabilistic model building genetic network programming using both of good and bad individuals , 2013 .

[7]  Shingo Mabu,et al.  Real Time Updating Genetic Network Programming for Adapting to the Change of Stock Prices , 2009 .

[8]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[9]  Qingbiao Meng,et al.  Genetic Network Programming with Sarsa Learning Based Nonuniform Mutation , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[10]  Shingo Mabu,et al.  Genetic Network Programming with Estimation of Distribution Algorithms for class association rule mining in traffic prediction , 2010, IEEE Congress on Evolutionary Computation.

[11]  Kotaro Hirasawa,et al.  Learning Petri network and its application to nonlinear system control , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Thomas Stützle,et al.  The Ant Colony Optimization Metaheuristic: Algorithms, Applications, and Advances , 2003 .

[13]  Shingo Mabu,et al.  A Graph-Based Evolutionary Algorithm: Genetic Network Programming (GNP) and Its Extension Using Reinforcement Learning , 2007, Evolutionary Computation.

[14]  Kotaro Hirasawa,et al.  Comparison between Genetic Network Programming (GNP) and Genetic Programming (GP) , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[15]  Prasanta K. Jana,et al.  A novel evolutionary approach for load balanced clustering problem for wireless sensor networks , 2013, Swarm Evol. Comput..

[16]  Seyed Naser Razavi,et al.  Empowering particle swarm optimization algorithm using multi agents' capability: A holonic approach , 2017, Knowl. Based Syst..

[17]  Shingo Mabu,et al.  Enhanced decision making mechanism of rule-based genetic network programming for creating stock trading signals , 2013, Expert Syst. Appl..

[18]  Shingo Mabu,et al.  Double-deck Elevator Group Supervisory Control System using Genetic Network Programming with Ant Colony Optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[19]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[20]  Leonardo Vanneschi,et al.  Multidimensional genetic programming for multiclass classification , 2019, Swarm Evol. Comput..

[21]  Hassan Khotanlou,et al.  Fuzzy evolutionary cellular learning automata model for text summarization , 2016, Swarm Evol. Comput..

[22]  Chittaranjan Tripathy,et al.  Fault tolerant scheduling of hard real-time tasks on multiprocessor system using a hybrid genetic algorithm , 2014, Swarm Evol. Comput..

[23]  D. Karaboga,et al.  On the performance of artificial bee colony (ABC) algorithm , 2008, Appl. Soft Comput..

[24]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[25]  Meihua Yang,et al.  Revisiting Genetic Network Programming (GNP): Towards the Simplified Genetic Operators , 2018, IEEE Access.

[26]  Amer Draa,et al.  A quantum-inspired genetic algorithm for solving the antenna positioning problem , 2016, Swarm Evol. Comput..

[27]  Samia Nefti-Meziani,et al.  A Comprehensive Review of Swarm Optimization Algorithms , 2015, PloS one.

[28]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[29]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[30]  Kotaro Hirasawa,et al.  Genetic Network Programming with Simplified Genetic Operators , 2013, ICONIP.

[31]  Armin Tavakoli Naeini,et al.  Improving coordination via emergent communication in cooperative multiagent systems: A Genetic Network Programming approach , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[32]  Meihua Yang,et al.  Niching genetic network programming with rule accumulation for decision making: An evolutionary rule-based approach , 2018, Expert Syst. Appl..

[33]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[34]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[35]  Kotaro Hirasawa,et al.  A study of evolutionary multiagent models based on symbiosis , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Akbar Esfahanipour,et al.  Generating ternary stock trading signals using fuzzy genetic network programming , 2016, 2016 Annual Conference of the North American Fuzzy Information Processing Society (NAFIPS).

[37]  Martha E. Pollack,et al.  Introducing the Tileworld: Experimentally Evaluating Agent Architectures , 1990, AAAI.

[38]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[39]  Shingo Mabu,et al.  Enhanced rule extraction and classification mechanism of genetic network programming for stock trading signal generation , 2011, GECCO '11.

[40]  Jinglu Hu,et al.  Elevator group control system using genetic network programming with ACO considering transitions , 2007, SICE Annual Conference 2007.

[41]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[42]  Shingo Mabu,et al.  An Intrusion-Detection Model Based on Fuzzy Class-Association-Rule Mining Using Genetic Network Programming , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[43]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[44]  Kenji Funahashi,et al.  Heterogeneous Multi-agents Learning Using Genetic Network Programming with Immune Adjustment Mechanism , 2009 .

[45]  Shingo Mabu,et al.  Robust Genetic Network Programming using SARSA Learning for autonomous robots , 2009, 2009 ICCAS-SICE.

[46]  Shingo Mabu,et al.  Genetic Network Programming with Reinforcement Learning Using Sarsa Algorithm , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[47]  Shingo Mabu,et al.  Evolving plural programs by genetic network programming with multi-start nodes , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[48]  Kotaro Hirasawa,et al.  Genetic Network Programming with Acquisition Mechanisms of Association Rules , 2006, J. Adv. Comput. Intell. Intell. Informatics.

[49]  Kotaro Hirasawa,et al.  Evolving directed graphs with artificial bee colony algorithm , 2014, 2014 14th International Conference on Intelligent Systems Design and Applications.

[50]  Shingo Mabu,et al.  A Novel Graph-Based Estimation of the Distribution Algorithm and its Extension Using Reinforcement Learning , 2014, IEEE Transactions on Evolutionary Computation.

[51]  Shingo Mabu,et al.  A novel estimation of distribution algorithm using graph-based chromosome representation and reinforcement learning , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[52]  Shingo Mabu,et al.  Online Learning of Genetic Network Programming , 2002 .

[53]  Mohammad Hamiruce Marhaban,et al.  Collision Prediction based Genetic Network Programming-Reinforcement Learning for Mobile Robot Navigation in Unknown Dynamic Environments , 2017 .