Control of the trajectory of a hexapod robot based on distributed Q-learning

This paper presents a distributed approach of reinforcement learning used to learn a hexapod robot to control its trajectory according to a multilevel control decomposition. Locomotion functionality which consists in coordinating the legs so as to assure stable gait and in controlling the posture of the robot is more particularly investigated. As any leg cannot achieve its movements without interacting with others, coordination problems may occur. In order to take into account the actions of other agents, a distributed version of Q learning is proposed. The amplitudes of the movements are coded by self-organising maps and are adjusted during the training stage. The results of the simulation show that the robot can learn to control its trajectory efficiently.

[1]  R. Full.  Integration of individual leg dynamics with whole body movement in arthropod locomotion , 1993 .

[2]  Andrew James Smith,et al.  Applications of the self-organising map to reinforcement learning , 2002, Neural Networks.

[3]  Roger D. Quinn,et al.  Posture control of a cockroach-like robot , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[4]  Enric Celaya,et al.  Control of a six-legged robot walking on abrupt terrain , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[5]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[6]  Enric Celaya,et al.  A control structure for the locomotion of a legged robot on difficult terrain , 1998, IEEE Robotics Autom. Mag..

[7]  P. Couturier,et al.  Distributed Reinforcement Learning of a Six-Legged Robot to Walk , 2003, 2003 4th International Conference on Control and Automation Proceedings.

[8]  Rodney A. Brooks,et al.  Fast, Cheap and Out of Control: a Robot Invasion of the Solar System , 1989 .