Interaction learning for dynamic movement primitives used in cooperative robotic tasks

Since several years dynamic movement primitives (DMPs) are more and more getting into the center of interest for flexible movement control in robotics. In this study we introduce sensory feedback together with a predictive learning mechanism which allows tightly coupled dual-agent systems to learn an adaptive, sensor-driven interaction based on DMPs. The coupled conventional (no-sensors, no learning) DMP-system automatically equilibrates and can still be solved analytically allowing us to derive conditions for stability. When adding adaptive sensor control we can show that both agents learn to cooperate. Simulations as well as real-robot experiments are shown. Interestingly, all these mechanisms are entirely based on low level interactions without any planning or cognitive component.

[1]  Oliver Kroemer,et al.  Adapting Preshaped Grasping Movements Using Vision Descriptors , 2010, SAB.

[2]  Ludovic Righetti,et al.  Programmable central pattern generators: an application to biped locomotion control , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[3]  A. Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[4]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[6]  Ludovic Righetti,et al.  Movement generation using dynamical systems : a humanoid robot performing a drumming task , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[7]  Andrej Gams,et al.  On-line periodic movement and force-profile learning for adaptation to new surfaces , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[8]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[9]  Stefan Schaal,et al.  Dynamics systems vs. optimal control--a unifying view. , 2007, Progress in brain research.

[10]  Florentin Wörgötter,et al.  Isotropic-sequence-order learning in a closed-loop behavioural system , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[11]  Ales Ude,et al.  Task adaptation through exploration and action sequencing , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[12]  S. Strogatz.  Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering , 1995 .

[13]  Masaru Uchiyama,et al.  Symmetric Kinematic Formulation and Non-Master/Slave Coordinated Control of Two Arm Robots , 1989 .

[14]  Toru Tsumugiwa,et al.  Variable impedance control with virtual stiffness for human-robot cooperative peg-in-hole task , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[16]  Aude Billard,et al.  BM: An iterative algorithm to learn stable non-linear dynamical systems with Gaussian mixture models , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Masaru Uchiyama,et al.  Hybrid position/Force control for coordination of a two-arm robot , 1987, Proceedings. 1987 IEEE International Conference on Robotics and Automation.

[18]  Ales Ude,et al.  Action sequencing using dynamic movement primitives , 2011, Robotica.

[19]  Abderrahmane Kheddar,et al.  Motion learning and adaptive impedance for robot control during physical interaction with humans , 2011, 2011 IEEE International Conference on Robotics and Automation.

[20]  Aude Billard,et al.  Imitation learning of globally stable non-linear point-to-point robot motions using nonlinear programming , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Jochen J. Steil,et al.  Interactive imitation learning of object movement skills , 2011, Autonomous Robots.

[22]  Ales Ude,et al.  Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives , 2011, Robotics Auton. Syst..

[23]  Jun Nakanishi,et al.  Learning Movement Primitives , 2005, ISRR.

[24]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[25]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[26]  Minija Tamosiunaite,et al.  Joining Movement Sequences: Modified Dynamic Movement Primitives for Robotics Applications Exemplified on Handwriting , 2012, IEEE Transactions on Robotics.

[27]  Torsten Bertram,et al.  Get Out of the Way – Obstacle Avoidance and Learning by Demonstration for Manipulation , 2011 .

[28]  Aude Billard,et al.  A dynamical system approach to realtime obstacle avoidance , 2012, Autonomous Robots.

[29]  Jun Morimoto,et al.  Learning parametric dynamic movement primitives from multiple demonstrations , 2011, Neural Networks.

[30]  Siddhartha S. Srinivasa,et al.  CHOMP: Gradient optimization techniques for efficient motion planning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[31]  Stefan Schaal,et al.  Online movement adaptation based on previous sensor experiences , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Stefan Schaal,et al.  Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[33]  Darwin G. Caldwell,et al.  Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  P. Chiacchio,et al.  Six-DOF Impedance Control of Dual-Arm Cooperative Manipulators , 2008, IEEE/ASME Transactions on Mechatronics.

[35]  Stefan Schaal,et al.  Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance , 2009, 2009 IEEE International Conference on Robotics and Automation.

[36]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[37]  A. A. Maciejewski,et al.  Obstacle Avoidance , 2005 .

[38]  Betty J. Mohler,et al.  Learning perceptual coupling for motor primitives , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[40]  Shigeru Kurono,et al.  Coordinated Computer Control of a Pair of Manipulators , 1975 .

[41]  Steven H. Strogatz,et al.  Nonlinear Dynamics and Chaos , 2024 .

[42]  E. Nakano,et al.  Cooperational Control of the Anthropomorphous Manipulator "MELARM" , 1974 .

[43]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[44]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[45]  G. Metta,et al.  A modular bio-inspired architecture for movement generation for the infant-like robot iCub , 2008, 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.