Reinforcement Learning for Robotic Assembly Using Non-Diagonal Stiffness Matrix

Contact-rich tasks, wherein multiple contact transitions occur in a series of operations, have been extensively studied for task automation. Precision assembly, a typical example of contact-rich tasks, requires high time constants to cope with the change in contact state. Therefore, this letter proposes a local trajectory planning method for precision assembly with high time constants. Because the non-diagonal component of a stiffness matrix can induce motion at high sampling frequencies, we use this concept to design a stiffness matrix to guide the motion of an object and propose a method to control it. We introduce reinforcement learning (RL) for the selection of the stiffness matrix because the relationship between the desired direction and the sensor response is difficult to model. An architecture with various sampling rates for RL and admittance control has the advantage of rapid response owing to a high time constant of the local trajectory modification. The effectiveness of the method is verified experimentally on two contact-rich tasks: inserting a peg into a hole and inserting a gear. Using the proposed method, the average total time needed to insert the peg in the hole is 1.64 s, which is less than half the time reported by the best of the existing state of the art studies.

[1]  Kyo Kutsuzawa,et al.  Admittance Control Based on a Stiffness Ellipse for Rapid Trajectory Deformation , 2020, 2020 IEEE 16th International Workshop on Advanced Motion Control (AMC).

[2]  Sergey Levine,et al.  Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Reward Signals , 2019 .

[3]  Stefan Schaal,et al.  Variable Impedance Control - A Reinforcement Learning Approach , 2010, Robotics: Science and Systems.

[4]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Razvan Pascanu,et al.  Sim-to-Real Robot Learning from Pixels with Progressive Nets , 2016, CoRL.

[6]  Oleg O. Sushkov,et al.  A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[7]  Kouhei Ohnishi,et al.  Motion control for advanced mechatronics , 1996 .

[8]  Takamitsu Matsubara,et al.  Learning Force Control for Contact-Rich Manipulation Tasks With Rigid Position-Controlled Robots , 2020, IEEE Robotics and Automation Letters.

[9]  Alice M. Agogino,et al.  Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[10]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[11]  Marcelo H. Ang,et al.  Specifying and achieving passive compliance based on manipulator structure , 1995, IEEE Trans. Robotics Autom..

[12]  Toshiaki Tsuji,et al.  Force control of a jumping musculoskeletal robot with pneumatic artificial muscles , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[14]  Sangcheol Lee,et al.  Development of a new variable remote center compliance (VRCC) with modified elastomer shear pad (ESP) for robot assembly , 2005, IEEE Transactions on Automation Science and Engineering.

[15]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation , 1984, 1984 American Control Conference.

[16]  Giovanni De Magistris,et al.  Deep reinforcement learning for high precision assembly tasks , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Ludovic Righetti,et al.  Learning Variable Impedance Control for Contact Sensitive Tasks , 2019, IEEE Robotics and Automation Letters.

[18]  Lucian Busoniu,et al.  Reinforcement learning for control: Performance, stability, and deep approximators , 2018, Annu. Rev. Control..

[19]  Ken Chen,et al.  Fast Skill Learning for Variable Compliance Robotic Assembly , 2019, ArXiv.

[20]  Young Loul Kim,et al.  Hole detection algorithm for chamferless square peg-in-hole based on shape recognition using F/T sensor , 2014 .

[21]  Homayoon Kazerooni,et al.  On the Stability of Robot Compliant Motion Control: Theory and Experiments , 1990 .

[22]  Prabir K. Pal,et al.  Intelligent and environment-independent Peg-In-Hole search strategies , 2013, 2013 International Conference on Control, Automation, Robotics and Embedded Systems (CARE).

[23]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[24]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[25]  Prashant Rao,et al.  Analyzing and Improving Cartesian Stiffness Control Stability of Series Elastic Tendon-Driven Robotic Hands , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Dale Lawrence,et al.  Position-based impedance control - Achieving stability in practice , 1987 .

[27]  Sergey Levine,et al.  Residual Reinforcement Learning for Robot Control , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[28]  Silvio Savarese,et al.  Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Ken Chen,et al.  Learning-based Variable Compliance Control for Robotic Assembly , 2018 .

[30]  Yu Zhao,et al.  Autonomous alignment of peg and hole by force/torque measurement for robotic assembly , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[31]  Silvio Savarese,et al.  Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[32]  Yiannis Karayiannidis,et al.  Robotic Assembly of Rounded Parts With and Without Threads , 2020, IEEE Robotics and Automation Letters.

[33]  J. Salisbury,et al.  Active stiffness control of a manipulator in cartesian coordinates , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.