A combined reactive and reinforcement learning controller for an autonomous tracked vehicle

Unmanned ground vehicles currently exhibit simple autonomous behaviours. This paper presents a control algorithm developed for a tracked vehicle to autonomously climb obstacles by varying its front and back track orientations. A reactive controller computes a desired geometric configuration based on terrain information. A reinforcement learning algorithm enhances vehicle mobility by finding effective exit strategies in deadlock situations. It is capable of incorporating complex information including terrain and vehicle dynamics through learned experiences. Experiments illustrate the effectiveness of the proposed approach for climbing various obstacles.

[1]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[2]  Michael Trentini,et al.  Perception and mobility research at Defence R&D Canada for UGVs in complex terrain , 2007, SPIE Defense + Commercial Sensing.

[3]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[4]  Leslie Pack Kaelbling,et al.  Automated Design of Adaptive Controllers for Modular Robots using Reinforcement Learning , 2008, Int. J. Robotics Res..

[5]  Michael Trentini,et al.  Intelligent mobility research for robotic locomotion in complex terrain , 2006, SPIE Defense + Commercial Sensing.

[6]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[7]  Rodney A. Brooks,et al.  Learning to Coordinate Behaviors , 1990, AAAI.

[8]  Alexander Kleiner,et al.  Behavior maps for online planning of obstacle negotiation and climbing on rough terrain , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Leslie Pack Kaelbling,et al.  Effective reinforcement learning for mobile robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[10]  Scott Kuindersma,et al.  Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories , 2010, NIPS.

[11]  Bruce Leonard Digney,et al.  Robotic concepts for urban operations , 2002, SPIE Defense + Commercial Sensing.

[12]  Stefan Schaal,et al.  Learning to Control in Operational Space , 2008, Int. J. Robotics Res..

[13]  Kaspar Althoefer,et al.  Track–terrain modelling and traversability prediction for tracked vehicles on soft terrain , 2010 .

[14]  Dean A. Pomerleau,et al.  Neural Network Vision for Robot Driving , 1997 .

[15]  Maja J. Mataric,et al.  Reward Functions for Accelerated Learning , 1994, ICML.

[16]  Chih-Han Yu,et al.  Quadruped robot obstacle negotiation via reinforcement learning , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[17]  Martin T. Hagan,et al.  Neural network design , 1995 .

[18]  Martin A. Riedmiller,et al.  Self-learning neural control of a mobile robot , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.