Control of Flexible Manipulator Based on Reinforcement Learning

Most researches about control of flexible manipulators are all based on the dynamic model, which is difficult to establish because of their flexibility and the tedious process of measuring flexible link's parameters. In this paper, the goal is to design a controller which is able to control the flexible manipulator to track a given position in joint space and suppress vibration without knowing the dynamic model. For the problem of tracking a given position, a tracking controller is designed based on sliding mode control, and for the purpose of vibration suppression, a vibration suppression controller is designed as a deep neural network. Because the input of the flexible manipulator, torques at each joint, is a high dimensional and continuous space, Deep Deterministic Policy Gradient Algorithm (DDPG) is adopted to train the neural network in the vibration suppression controller. The effectiveness of the proposed controller to track a given position and suppress vibration is demonstrated by numerical simulation.

[1]  Frank L. Lewis,et al.  Implementation of a neural network tracking controller for a single flexible link: comparison with PD and PID controllers , 1998, IEEE Trans. Ind. Electron..

[2]  Fumihito Arai,et al.  Modeling and Vibration Control with Neural Network for Flexible Multi-Link Structure. , 1999 .

[3]  Bidyadhar Subudhi,et al.  Real-Time Adaptive Control of a Flexible Manipulator Using Reinforcement Learning , 2012, IEEE Transactions on Automation Science and Engineering.

[4]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[5]  Y. Istefanopulos,et al.  End-effector Trajectory Control in a Two-Link Flexible Manipulator Through Reference Joint Angle Values Modification by Neural Networks , 2006 .

[6]  Shuzhi Sam Ge,et al.  Vibration control of an Euler–Bernoulli beam under unknown spatiotemporally varying disturbance , 2011, Int. J. Control.

[7]  Dong Liang,et al.  Singular-Perturbation-Based Nonlinear Hybrid Control of Redundant Parallel Robot , 2018, IEEE Transactions on Industrial Electronics.

[8]  R. Y. Yakoub,et al.  Three Dimensional Absolute Nodal Coordinate Formulation for Beam Elements: Implementation and Applications , 2001 .

[9]  Changyin Sun,et al.  Neural Network Control of a Flexible Robotic Manipulator Using the Lumped Spring-Mass Model , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[10]  R. Y. Yakoub,et al.  Three Dimensional Absolute Nodal Coordinate Formulation for Beam Elements: Theory , 2001 .

[11]  Bruno Siciliano,et al.  A Singular Perturbation Approach to Control of Lightweight Flexible Manipulators , 1988, Int. J. Robotics Res..

[12]  ChangSu Ha,et al.  Passivity-based control of manipulator-stage systems on vertical flexible beam , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[14]  Wei He,et al.  Reinforcement learning control of a single-link flexible robotic manipulator , 2017 .

[15]  Liang Zhao,et al.  Optimal Trajectory Planning for Manipulators with Flexible Curved Links , 2016, IAS.

[16]  Johannes Gerstmayr,et al.  Analysis of Thin Beams and Cables Using the Absolute Nodal Co-ordinate Formulation , 2006 .

[17]  Li Lin,et al.  Modeling and simulation of EAST flexible in-vessel inspection robot based on absolute nodal coordinate formulation , 2014, 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014).

[18]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[19]  C. Liu,et al.  New spatial curved beam and cylindrical shell elements of gradient-deficient Absolute Nodal Coordinate Formulation , 2012 .

[20]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .