An Extension of Genetic Network Programming with Reinforcement Learning Using Actor-Critic

A new graph-based evolutionary algorithm named "Genetic Network Programming, GNP" has been already proposed. GNP represents its solutions as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) was proposed a few years ago. Since GNP-RL can do reinforcement learning during task execution in addition to evolution after task execution, it can search for solutions efficiently. In this paper, GNP with Actor-Critic (GNP-AC) which is a new type of GNP-RL is proposed. Originally, GNP deals with discrete information, but GNP-AC aims to deal with continuous information. The proposed method is applied to the controller of the Khepera simulator and its performance is evaluated.

[1]  Shingo Mabu,et al.  Genetic network programming with learning and evolution for adapting to dynamical environments , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[2]  Shingo Mabu,et al.  Online Learning of Genetic Network Programming , 2002 .

[3]  Kotaro Hirasawa,et al.  A study of evolutionary multiagent models based on symbiosis , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[5]  Shingo Mabu,et al.  Online learning of genetic network programming (GNP) , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[6]  Shingo Mabu,et al.  Genetic Network Programming with Reinforcement Learning and Its Performance Evaluation , 2004, GECCO.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.