Manipulation Skill Acquisition for Robotic Assembly using Deep Reinforcement Learning

Nowadays mobile manipulators are commonly used in assembly tasks, which can reach greater workspace but also cause more uncertainties. Uncertainty is one of the main factors affecting the quality and efficiency of assembly tasks. The framework of skill acquisition based on machine learning can make a good job of solving the uncertainties in mobile manipulation. In this paper, a method based on deep reinforcement learning is proposed to learn assembly skills. The Deep Deterministic Policy Gradient algorithm based on off-policy can scale to complex manipulation tasks. The safety constraint was set to the torque of each joint. The positive and negative reward function is designed to improve learning efficiency, which could be used in different assembly tasks. The deep neural network policies could be trained out efficiently on a real robot platform. This approach was performed on the KUKA iiwa robot arm equipped with 7-dof force/torque sensors. The results show that the robot could learn assembly skills properly with no prior knowledge.

[1]  Sergey Levine,et al.  QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation , 2018, CoRL.

[2]  Meng Wei,et al.  Robot skill acquisition in assembly process using deep reinforcement learning , 2019, Neurocomputing.

[3]  Yibin Li,et al.  Contact State Classification in Industrial Robotic Assembly Tasks Based on Extreme Learning Machine , 2018, 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER).

[4]  Peter Plapper,et al.  Contact-state modeling of robotic assembly tasks using Gaussian mixture models , 2014 .

[5]  Yunhua Li,et al.  Assembly states identification for electromechanical actuator based on motor current signal analysis , 2017, 2017 12th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[6]  Kerstin Küsters.  See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion , 2019 .

[7]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[8]  John Norrish,et al.  Recent Progress on Programming Methods for Industrial Robots , 2010, ISR/ROBOTIK.

[9]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[10]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[11]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[12]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Rüdiger Dillmann,et al.  Teaching and learning of robot tasks via observation of human performance , 2004, Robotics Auton. Syst..

[14]  Giovanni De Magistris,et al.  Deep reinforcement learning for high precision assembly tasks , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Sergey Levine,et al.  Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection , 2016, ISER.

[16]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[18]  Stefan Schaal,et al.  Learning force control policies for compliant manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[20]  Ismael Lopez-Juarez,et al.  Towards learning contact states during peg-in-hole assembly with a dual-arm robot , 2017, 2017 CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON).

[21]  R. Mazo.  On the theory of brownian motion , 1973 .

[22]  Weihua Sheng,et al.  Automated assembly skill acquisition and implementation through human demonstration , 2018, Robotics Auton. Syst..