Efficient learning variable impedance control for industrial robots

Compared with the robots, humans can learn to perform various contact tasks in unstructured environments by modulating arm impedance characteristics. In this article, we consider endowing this compliant ability to the industrial robots to effectively learn to perform repetitive force-sensitive tasks. Current learning impedance control methods usually suffer from inefficiency. This paper establishes an efficient variable impedance control method. To improve the learning efficiency, we employ the probabilistic Gaussian process model as the transition dynamics of the system for internal simulation, permitting long-term inference and planning in a Bayesian manner. Then, the optimal impedance regulation strategy is searched using a model-based reinforcement learning algorithm. The effectiveness and efficiency of the proposed method are verified through force control tasks using a 6-DoFs Reinovo industrial manipulator.

[1]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation: Part I—Theory , 1985 .

[2]  R A Scheidt,et al.  Impedance control and internal model formation when reaching in a randomly varying dynamical environment. , 2001, Journal of neurophysiology.

[3]  Yoshiyuki Tanaka,et al.  On-line Learning of Robot Arm Impedance Using Neural Networks , 2004, 2004 IEEE International Conference on Robotics and Biomimetics.

[4]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[5]  Rieko Osu,et al.  CNS Learns Stable, Accurate, and Efficient Movements Using a Simple Algorithm , 2008, The Journal of Neuroscience.

[6]  Martin Buss,et al.  Force Tracking Impedance Control with Variable Target Stiffness , 2008 .

[7]  Jan Peters,et al.  Learning motor primitives for robotics , 2009, 2009 IEEE International Conference on Robotics and Automation.

[8]  Sethu Vijayakumar,et al.  Exploiting sensorimotor stochasticity for learning control of variable impedance actuators , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[9]  Marc Peter Deisenroth,et al.  Efficient reinforcement learning using Gaussian processes , 2010 .

[10]  Alin Albu-Schäffer,et al.  Human-Like Adaptation of Force and Impedance in Stable and Unstable Interactions , 2011, IEEE Transactions on Robotics.

[11]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[12]  Stefan Schaal,et al.  Learning variable impedance control , 2011, Int. J. Robotics Res..

[13]  Sethu Vijayakumar,et al.  Learning impedance control of antagonistic systems based on stochastic optimization principles , 2011, Int. J. Robotics Res..

[14]  Stefan Schaal,et al.  Learning force control policies for compliant manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Alin Albu-Schäffer,et al.  A versatile biomimetic controller for contact tooling and haptic exploration , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  Stefan Schaal,et al.  Model-Free Reinforcement Learning of Impedance Control in Stochastic Environments , 2012, IEEE Transactions on Autonomous Mental Development.

[17]  Sethu Vijayakumar,et al.  Optimal variable stiffness control: formulation and application to explosive movement tasks , 2012, Auton. Robots.

[18]  Cristian Secchi,et al.  A tank-based approach to impedance control with variable stiffness , 2013, 2013 IEEE International Conference on Robotics and Automation.

[19]  Krzysztof Jaskot,et al.  The dynamics of the human arm with an observer for the capture of body motion parameters , 2013 .

[20]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[21]  Auke Jan Ijspeert,et al.  Exploiting natural dynamics in biped locomotion using variable impedance control , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[22]  Aude Billard,et al.  Learning object-level impedance control for robust grasping and dexterous manipulation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Aude Billard,et al.  Learning Compliant Manipulation through Kinesthetic and Tactile Human-Robot Interaction , 2014, IEEE Transactions on Haptics.

[24]  Carl E. Rasmussen,et al.  Gaussian Processes for Data-Efficient Learning in Robotics and Control , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[26]  Sandra Hirche,et al.  Generalization of Force Control Policies from Demonstrations for Constrained Robotic Motion Tasks , 2015, J. Intell. Robotic Syst..

[27]  Dongheui Lee,et al.  The Role of Coupling Terms in Variable Impedance Policies Learning , 2016 .

[28]  Oussama Khatib,et al.  Learning potential functions from human demonstrations with encapsulated dynamic and compliant behaviors , 2017, Auton. Robots.

[29]  Athanasios S. Polydoros,et al.  Survey of Model-Based Reinforcement Learning: Applications on Robotics , 2017, J. Intell. Robotic Syst..

[30]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[31]  Misha Denil,et al.  Learning to Perform Physics Experiments via Deep Reinforcement Learning , 2016, ICLR.

[32]  Wei Wang,et al.  Variable Admittance Control Based on Fuzzy Reinforcement Learning for Minimally Invasive Surgery Manipulator , 2017, Sensors.

[33]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[34]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Zhi Liu,et al.  Personalized Variable Gain Control With Tremor Attenuation for Robot Teleoperation , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[36]  Sergey Levine,et al.  Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).