Real Time Robot Policy Adaptation Based on Intelligent Algorithms

In this paper we present a new method for robot real time policy adaptation by combining learning and evolution. The robot adapts the policy as the environment conditions change. In our method, we apply evolutionary computation to find the optimal relation between reinforcement learning parameters and robot performance. The proposed algorithm is evaluated in the simulated environment of the Cyber Rodent (CR) robot, where the robot has to increase its energy level by capturing the active battery packs. The CR robot lives in two environments with different settings that replace each other four times. Results show that evolution can generate an optimal relation between the robot performance and exploration-exploitation of reinforcement learning, enabling the robot to adapt online its strategy as the environment conditions change.

[1]  Tohgoroh Matsui.  Adapting to Subsequent Changes of Environment by Learning Policy Preconditions , 2002 .

[2]  Kenji Doya,et al.  Evolution of meta-parameters in reinforcement learning algorithm , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[3]  Genci Capi,et al.  Multiobjective Evolution of Neural Controllers and Task Complexity , 2007, IEEE Transactions on Robotics.

[4]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[5]  Kenji Doya,et al.  Reinforcement Learning in Continuous Time and Space , 2000, Neural Computation.

[6]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[7]  Rodney A. Brooks,et al.  Artificial Life IV: Proceedings of the Fourth International Workshop on the Synthesis and Simlulation of Living Systmes , 1994 .

[8]  Takashi Minato,et al.  Environmental Change Adaptation for Mobile Robot Navigation , 2000 .

[9]  R. French,et al.  Genes, Phenes and the Baldwin Effect: Learning and Evolution in a Simulated Population , 1994 .

[10]  Y. Niv,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002 .

[11]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[12]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[13]  Kenji Doya,et al.  APPLICATION OF EVOLUTIONARY COMPUTATION FOR EFFICIENT REINFORCEMENT LEARNING , 2006, Appl. Artif. Intell..

[14]  Charles E. Taylor,et al.  Artificial Life II , 1991 .

[15]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[16]  Stan Matwin,et al.  INFERRING AND REVISING THEORIES WITH CONFIDENCE: ANALYZING BILINGUALISM IN THE 1901 CANADIAN CENSUS , 2006, Appl. Artif. Intell..