Learning potential functions from human demonstrations with encapsulated dynamic and compliant behaviors

We consider the problem of devising a unified control policy capable of regulating both the robot motion and its physical interaction with the environment. We formulate this control policy by a non-parametric potential function and a dissipative field, which both can be learned from human demonstrations. We show that the robot motion and its stiffness behaviors can be encapsulated by the potential function’s gradient and curvature, respectively. The dissipative field can also be used to model desired damping behavior throughout the motion, hence generating motions that follows the same velocity profile as the demonstrations. The proposed controller can be realized as a unification approach between “realtime motion generation” and “variable impedance control”, with the advantages that it has guaranteed stability as well as does not rely on following a reference trajectory. Our approach, called unified motion and variable impedance control (UMIC), is completely time-invariant and can be learned from a few demonstrations via solving two (convex) constrained quadratic optimization problems. We validate UMIC on a library of 30 human handwriting motions and on a set of experiments on 7-DoF KUKA light weight robot.

[1]  Sungchul Kang,et al.  Impedance Learning for Robotic Contact Tasks Using Natural Actor-Critic Algorithm , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[3]  Oliver Kroemer,et al.  Learning to select and generalize striking movements in robot table tennis , 2012, AAAI Fall Symposium: Robots Learning Interactively from Human Teachers.

[4]  Eric L. Sauser,et al.  An Approach Based on Hidden Markov Model and Gaussian Mixture Regression , 2010 .

[5]  Cristian Secchi,et al.  A tank-based approach to impedance control with variable stiffness , 2013, 2013 IEEE International Conference on Robotics and Automation.

[6]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Darwin G. Caldwell,et al.  Learning and Reproduction of Gestures by Imitation , 2010, IEEE Robotics & Automation Magazine.

[8]  Moshe Cohen,et al.  Learning impedance parameters for robot control using an associative search network , 1991, IEEE Trans. Robotics Autom..

[9]  Daniel E. Koditschek,et al.  Robot planning and control via potential functions , 1989 .

[10]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[11]  Aude Billard,et al.  A dynamical system approach to realtime obstacle avoidance , 2012, Autonomous Robots.

[12]  Aude Billard,et al.  Learning Compliant Manipulation through Kinesthetic and Tactile Human-Robot Interaction , 2014, IEEE Transactions on Haptics.

[13]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[14]  Aude Billard,et al.  Handbook of Robotics Chapter 59 : Robot Programming by Demonstration , 2007 .

[15]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[16]  Aude Billard,et al.  Modeling robot discrete movements with state-varying stiffness and damping: A framework for integrated motion generation and impedance control , 2014, Robotics: Science and Systems.

[17]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation , 1984, 1984 American Control Conference.

[18]  Yoji Yamada,et al.  The role of joint stiffness enhancing collision reaction performance of collaborative robot manipulators , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Anthony G. Pipe,et al.  An Architecture for Learning "Potential Field" Cognitive Maps with an Application to Mobile Robotics , 2000, Adapt. Behav..

[20]  Martin Buss,et al.  Force Tracking Impedance Control with Variable Target Stiffness , 2008 .

[21]  John Kenneth Salisbury,et al.  Playing it safe [human-friendly robots] , 2004, IEEE Robotics & Automation Magazine.

[22]  Luis Moreno,et al.  Kinesthetic teaching via Fast Marching Square , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Oussama Khatib,et al.  A Unified Framework for Whole-Body Humanoid Robot Control with Multiple Constraints and Contacts , 2008, EUROS.

[24]  Luigi Villani,et al.  Handbook of Robotics Chapter 7 : Force Control , 2007 .

[25]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[26]  G. Hirzinger,et al.  A new variable stiffness design: Matching requirements of the next robot generation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[27]  Stefan Schaal,et al.  Learning variable impedance control , 2011, Int. J. Robotics Res..

[28]  Stefan Schaal,et al.  Model-Free Reinforcement Learning of Impedance Control in Stochastic Environments , 2012, IEEE Transactions on Autonomous Mental Development.

[29]  Daniel E. Koditschek,et al.  Exact robot navigation using artificial potential functions , 1992, IEEE Trans. Robotics Autom..

[30]  Aude Billard,et al.  Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions , 2014, Robotics Auton. Syst..

[31]  Sethu Vijayakumar,et al.  Methods for Learning Control Policies from Variable-Constraint Demonstrations , 2010, From Motor Learning to Interaction Learning in Robots.

[32]  Seyed Mohammad,et al.  A Dynamical System-based Approach to Modeling Stable Robot Control Policies via Imitation Learning , 2012 .

[33]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[34]  Christian Ott,et al.  Cartesian Impedance Control of Redundant and Flexible-Joint Robots , 2008, Springer Tracts in Advanced Robotics.

[35]  Sethu Vijayakumar,et al.  Transferring Human Impedance Behavior to Heterogeneous Variable Impedance Actuators , 2013, IEEE Transactions on Robotics.

[36]  Darwin G. Caldwell,et al.  Encoding the time and space constraints of a task in explicit-duration Hidden Markov Model , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Aude Billard,et al.  Learning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators , 2011, Int. J. Robotics Res..

[38]  Weiping Li,et al.  Applied Nonlinear Control , 1991 .

[39]  Alessandro De Luca,et al.  Collision detection and reaction: A contribution to safe physical Human-Robot Interaction , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Oussama Khatib,et al.  Inertial Properties in Robotic Manipulation: An Object-Level Framework , 1995, Int. J. Robotics Res..

[41]  Darwin G. Caldwell,et al.  Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[43]  Alin Albu-Schäffer,et al.  A versatile biomimetic controller for contact tooling and haptic exploration , 2012, 2012 IEEE International Conference on Robotics and Automation.

[44]  Pradeep K. Khosla,et al.  Real-time obstacle avoidance using harmonic potential functions , 1991, IEEE Trans. Robotics Autom..

[45]  Stephen P. Boyd,et al.  CVXGEN: a code generator for embedded convex optimization , 2011, Optimization and Engineering.

[46]  A. Billard,et al.  Learning the Nonlinear Multivariate Dynamics of Motion of Robotic Manipulators , 2009 .

[47]  Sethu Vijayakumar,et al.  Learning impedance control of antagonistic systems based on stochastic optimization principles , 2011, Int. J. Robotics Res..

[48]  Aude Billard,et al.  Learning object-level impedance control for robust grasping and dexterous manipulation , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[49]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).