PPMC RL Training Algorithm: Rough Terrain Intelligent Robots through Reinforcement Learning.

Robots can now learn how to make decisions and control themselves, generalizing learned behaviors to unseen scenarios. In particular, AI powered robots show promise in rough environments like the lunar surface, due to the environmental uncertainties. We address this critical generalization aspect for robot locomotion in rough terrain through a training algorithm we have created called the Path Planning and Motion Control (PPMC) Training Algorithm. This algorithm is coupled with any generic reinforcement learning algorithm to teach robots how to respond to user commands and to travel to designated locations on a single neural network. In this paper, we show that the algorithm works independent of the robot structure, demonstrating that it works on a wheeled rover in addition the past results on a quadruped walking robot. Further, we take several big steps towards real world practicality by introducing a rough highly uneven terrain. Critically, we show through experiments that the robot learns to generalize to new rough terrain maps, retaining a 100% success rate. To the best of our knowledge, this is the first paper to introduce a generic training algorithm teaching generalized PPMC in rough environments to any robot, with just the use of reinforcement learning.

[1]  Sunme Kim,et al.  Brain-inspired Method for Hyper-connected and Distributed Intelligence , 2020, 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[2]  Kazuya Yoshida,et al.  High-speed mobility on planetary surfaces: A technical review , 2019, J. Field Robotics.

[3]  Kostas E. Bekris,et al.  Adaptive Tensegrity Locomotion on Rough Terrain via Reinforcement Learning , 2018, ArXiv.

[4]  Erwin Coumans,et al.  Bullet physics simulation , 2015, SIGGRAPH Courses.

[5]  Sergey Levine,et al.  Residual Reinforcement Learning for Robot Control , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[6]  Shoya Higa,et al.  Vision-Based Estimation of Driving Energy for Planetary Rovers Using Deep Learning and Terramechanics , 2019, IEEE Robotics and Automation Letters.

[7]  Ming Liu,et al.  Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[8]  Yuval Tassa,et al.  Emergence of Locomotion Behaviours in Rich Environments , 2017, ArXiv.

[9]  Tamir Blum,et al.  Adaptive Slope Locomotion with Deep Reinforcement Learning , 2020, 2020 IEEE/SICE International Symposium on System Integration (SII).

[10]  Atil Iscen,et al.  Data Efficient Reinforcement Learning for Legged Robots , 2019, CoRL.

[11]  Yanbin Gao,et al.  Towards Goal-Directed Navigation Through Combining Learning Based Global and Local Planners , 2019, Sensors.

[12]  John Walker,et al.  Initial Design Characteristics, Testing and Performance Optimisation for a Lunar Exploration Micro-Rover Prototype , 2018, Advances in Astronautics Science and Technology.

[13]  Masahiro Ono,et al.  Fast approximate clearance evaluation for rovers with articulated suspension systems , 2018, J. Field Robotics.

[14]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[15]  Atil Iscen,et al.  Sim-to-Real: Learning Agile Locomotion For Quadruped Robots , 2018, Robotics: Science and Systems.

[16]  Tamir Blum,et al.  PPMC Training Algorithm: A Deep Learning Based Path Planner and Motion Controller , 2020, 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[17]  Surya P. N. Singh,et al.  V-REP: A versatile and scalable robot simulation framework , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Elman Mansimov,et al.  Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation , 2017, NIPS.

[19]  Gabriel Dulac-Arnold,et al.  Challenges of Real-World Reinforcement Learning , 2019, ArXiv.