Motion Control of Non-Holonomic Constrained Mobile Robot Using Deep Reinforcement Learning

For the motion control problem of non-holonomic constrained mobile robots, a point stabilization kinematic control law for mobile robot based on deep reinforcement learning is proposed. Firstly, a kinematic model of mobile robot is constructed to build memory for deep reinforcement learning, including the current state of the robot, the control action, the reward and the next state of the robot, which is generated through the connection between mobile robot and environment. Then, value network parameters in the real-time network are updated by a loss function, which is composed of a state-action value in current moment came from the value network of real-time network and a target value, the state-action value of next moment generated by the value network in target network. Next, the parameters of policy network of real-time network are updated according to the state-action value generated by value network of the real-time network in current moment. Finally, the parameters in the real-time network are weighted and averaged with the parameters in the target network, so the parameters of target network are updated to control mobile robot to stabilize with desired point. The simulation and experiment results show that the control algorithm based on deep reinforcement learning could effectively realize the point stabilization control of nonholonomic mobile robots.

[1]  R. W. Brockett,et al.  Asymptotic stability and feedback stabilization , 1982 .

[2]  C. L. Philip Chen,et al.  A survey of human-centered intelligent robots: issues and challenges , 2017, IEEE/CAA Journal of Automatica Sinica.

[3]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[4]  Yingchong Ma,et al.  Control of nonholonomic wheeled mobile robots via i-PID controller , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[7]  M. Michaek,et al.  Vector-Field-Orientation Feedback Control Method for a Differentially Driven Vehicle , 2010, IEEE Transactions on Control Systems Technology.

[8]  Yaonan Wang,et al.  Simultaneous Stabilization and Tracking of Nonholonomic Mobile Robots: A Lyapunov-Based Approach , 2015, IEEE Transactions on Control Systems Technology.

[9]  Kui Li,et al.  Finite-time tracking control for extended nonholonomic chained-form systems with parametric uncertainty and external disturbance , 2018 .

[10]  Hussein A. Abbass,et al.  Hierarchical Deep Reinforcement Learning for Continuous Action Control , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[12]  Guoqiang Hu,et al.  Model-Free Unified Tracking and Regulation Visual Servoing of Wheeled Mobile Robots , 2016, IEEE Transactions on Control Systems Technology.

[13]  Krzysztof Kozlowski,et al.  Feedback control framework for car-like robots using the unicycle controllers , 2012, Robotica.

[14]  Sen Wang,et al.  Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning , 2018, Robotics Auton. Syst..

[15]  Hicham Chaoui,et al.  Linear Time-varying feedback Law for vehicles with Ackermann steering , 2017, Int. J. Robotics Autom..

[16]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[17]  S. Ali A. Moosavian,et al.  Switching Control of a Tractor-trailer wheeled robot , 2015, Int. J. Robotics Autom..

[18]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[19]  Sergey Levine,et al.  Deep Reinforcement Learning for Robotic Manipulation , 2016, ArXiv.