Autonomous control of real snake-like robot using reinforcement learning; Abstraction of state-action space using properties of real world

In this paper we consider autonomous control of a real snake-like robot using reinforcement learning. We focus on curse of dimensionality and lack of generality, and point out that the causes of the problems are not in learning algorithm but in neglect of properties of the real world. To solve the problems we propose new framework in which body of robot abstract general meaning by using properties of the real world as information processor. We apply the proposed framework for controlling a snake-like robot and confirm that the two problems are solved simultaneously, without changing learning algorithm at all. To demonstrate the effectiveness of the proposed framework experiments has been carried out.

[1]  Shigenobu Kobayashi,et al.  Reinforcement learning of walking behavior for a four-legged robot , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[2]  Masaki Ogino,et al.  A Humanoid Approaches to the Goal - Reinforcement Learning Based on Rhythmic Walking Parameters , 2003, RoboCup.

[3]  Kenji Doya,et al.  Neural mechanisms of learning and control , 2001 .

[4]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[5]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.

[6]  Suguru Arimoto,et al.  A New Feedback Method for Dynamic Control of Manipulators , 1981 .

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Kazuyuki Ito,et al.  Autonomous control of a snake-like robot utilizing passive mechanism , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  Suguru Arimoto.  What are the Fundamentals of Bio-Mimetic Control? , 2004, 2004 IEEE International Conference on Robotics and Biomimetics.

[11]  Larry D. Pyeatt,et al.  Decision Tree Function Approximation in Reinforcement Learning , 1999 .

[12]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.