Deep Reinforcement Learning for Robotic Assembly of Mixed Deformable and Rigid Objects

Reinforcement learning for assembly tasks can yield powerful robot control algorithms for applications that are challenging or even impossible for “conventional” feedback control methods. Insertion of a rigid peg into a deformable hole of smaller diameter is such a task. In this contribution we solve this task with Deep Reinforcement Learning. Force-torque measurements from a robot arm wrist sensor are thereby incorporated two-fold; they are integrated into the policy learning process and they are exploited in an admittance controller that is coupled to the neural network. This enables robot learning of contact-rich assembly tasks without explicit joint torque control or passive mechanical compliance. We demonstrate our approach in experiments with an industrial robot.

[1]  Daniel E. Whitney,et al.  Historical Perspective and State of the Art in Robot Force Control , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[2]  Roderic A. Grupen,et al.  Learning reactive admittance control , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[5]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[6]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[7]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[8]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[9]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[10]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[11]  Sergey Levine,et al.  Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics , 2014, NIPS.

[12]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[13]  Masayoshi Tomizuka,et al.  A Learning-Based Framework for Robot Peg-Hole-Insertion , 2015, HRI 2015.

[14]  Nolan Wagener,et al.  Learning contact-rich manipulation skills with guided policy search , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Ross A. Knepper,et al.  DeepMPC: Learning Deep Latent Features for Model Predictive Control , 2015, Robotics: Science and Systems.

[16]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[17]  Sergey Levine,et al.  Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Yu Zhao,et al.  Autonomous alignment of peg and hole by force/torque measurement for robotic assembly , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[19]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[20]  Sergey Levine,et al.  Guided Policy Search via Approximate Mirror Descent , 2016, NIPS.

[21]  Gaurav S. Sukhatme,et al.  Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning , 2017, ICML.

[22]  Sergey Levine,et al.  Path integral guided policy search , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Giovanni De Magistris,et al.  Deep reinforcement learning for high precision assembly tasks , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Alice M. Agogino,et al.  Tensegrity Robot Locomotion Under Limited Sensory Inputs via Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).