A novel estimation of distribution algorithm using graph-based chromosome representation and reinforcement learning

This paper proposed a novel EDA, where a directed graph network is used to represent its chromosome. In the proposed algorithm, a probabilistic model is constructed from the promising individuals of the current generation using reinforcement learning, and used to produce the new population. The node connection probability is studied to develop the probabilistic model, therefore pairwise interactions can be demonstrated to identify and recombine building blocks in the proposed algorithm. The proposed algorithm is applied to a problem of agent control, i.e., autonomous robot control. The experimental results show the superiority of the proposed algorithm comparing with the conventional algorithms.

[1]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[2]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[3]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[4]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[5]  Kotaro Hirasawa,et al.  A study of evolutionary multiagent models based on symbiosis , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[7]  Hisashi Handa.  EDA-RL: estimation of distribution algorithms for reinforcement learning problems , 2009, GECCO '09.

[8]  Hussein A. Abbass,et al.  A Survey of Probabilistic Model Building Genetic Programming , 2006, Scalable Optimization via Probabilistic Modeling.

[9]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[10]  Hitoshi Iba,et al.  A Bayesian Network Approach to Program Generation , 2008, IEEE Transactions on Evolutionary Computation.

[11]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[12]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[14]  Qingbiao Meng,et al.  Genetic Network Programming with Sarsa Learning Based Nonuniform Mutation , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[15]  Astro Teller,et al.  PADO: Learning Tree Structured Algorithms for Orchestration into an Object Recognition System , 1995 .

[16]  Kotaro Hirasawa,et al.  A Double-Deck Elevator Group Supervisory Control System Using Genetic Network Programming , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[17]  Jinglu Hu,et al.  Genetic network programming - application to intelligent agents , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[18]  Shingo Mabu,et al.  A Graph-Based Evolutionary Algorithm: Genetic Network Programming (GNP) and Its Extension Using Reinforcement Learning , 2007, Evolutionary Computation.

[19]  Kotaro Hirasawa,et al.  Comparison between Genetic Network Programming (GNP) and Genetic Programming (GP) , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[20]  H. Iba,et al.  Estimation of distribution programming based on Bayesian network , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[21]  D. Goldberg,et al.  Probabilistic Model Building and Competent Genetic Programming , 2003 .

[22]  T. Murata,et al.  Multi-agent Cooperation Using Genetic Network Programming with Automatically Defined Groups , 2004, GECCO.

[23]  Julian Francis Miller,et al.  Cartesian genetic programming , 2010, GECCO.

[24]  Kotaro Hirasawa,et al.  Genetic Network Programming with Acquisition Mechanisms of Association Rules , 2006, J. Adv. Comput. Intell. Intell. Informatics.

[25]  Peter Nordin,et al.  Evolution of a world model for a miniature robot using genetic programming , 1998, Robotics Auton. Syst..

[26]  Shingo Mabu,et al.  Genetic Network Programming with Estimation of Distribution Algorithms for class association rule mining in traffic prediction , 2010, IEEE Congress on Evolutionary Computation.