Adaptation and coaching of periodic motion primitives through physical and visual interaction

In this paper we propose and evaluate a control system to (1) learn and (2) adapt robot motion for continuous non-rigid contact with the environment. We present the approach in the context of wiping surfaces with robots. Our approach is based on learning by demonstration. First an initial periodic motion, covering the essence of the wiping task, is transferred from a human to a robot. The system extracts and learns one period of motion. Once the user/demonstrator is content with the motion, the robot seeks and establishes contact with a given surface, maintaining a predefined force of contact through force feedback. The shape of the surface is encoded for the complete period of motion, but the robot can adapt to a different surface, perturbations or obstacles. The novelty stems from the fact that the feedforward component is learned and encoded in a dynamic movement primitive. By using the feedforward component, the feedback component is greatly reduced if not completely canceled. Finally, if the user is not satisfied with the periodic pattern, he/she can change parts of motion through predefined gestures or through physical contact in a manner of a tutor or a coach.The complete system thus allows not only a transfer of motion, but a transfer of motion with matching correspondences, i.e. wiping motion is constrained to maintain physical contact with the surface to be wiped. The interface for both learning and adaptation is simple and intuitive and allows for fast and reliable knowledge transfer to the robot.Simulated and real world results in the application domain of wiping a surface are presented on three different robotic platforms. Results of the three robotic platforms, namely a 7 degree-of-freedom Kuka LWR-4 robot, the ARMAR-IIIa humanoid platform and the Sarcos CB-i humanoid robot, depict different methods of adaptation to the environment and coaching. An intuitive and user friendly system for transferring of skills from a person to a robot.It allows online learning and adaptation of motion trajectories.It allows adaptation of trajectories through human coaching, from either force or visual feedback.It is based on the dynamic motion primitives framework.Surface wiping use-case through non-rigid contact is demonstrated and evaluated.

[1]  Jun Morimoto,et al.  Online approach for altering robot behaviors based on human in the loop coaching gestures , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Aude Billard,et al.  Imitation learning of globally stable non-linear point-to-point robot motions using nonlinear programming , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Ales Ude,et al.  Learning to pour with a robot arm combining goal and shape learning for dynamic movement primitives , 2011, Robotics Auton. Syst..

[4]  Stefan Schaal,et al.  Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance , 2009, 2009 IEEE International Conference on Robotics and Automation.

[5]  Nikolaos G. Tsagarakis,et al.  Statistical dynamical systems for skills acquisition in humanoids , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[6]  Valerio Ortenzi,et al.  An experimental study of robot control during environmental contacts based on projected operational space dynamics , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[7]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[8]  Stefan Schaal,et al.  Reinforcement Learning With Sequences of Motion Primitives for Robust Manipulation , 2012, IEEE Transactions on Robotics.

[9]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[10]  Carme Torras,et al.  A robot learning from demonstration framework to perform force-based manipulation tasks , 2013, Intelligent Service Robotics.

[11]  Sandra Hirche,et al.  Learning interaction control policies by demonstration , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Stefan Schaal,et al.  Encoding of periodic and their transient motions by a single dynamic movement primitive , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[13]  Aude Billard,et al.  Iterative learning of grasp adaptation through human corrections , 2012, Robotics Auton. Syst..

[14]  Rüdiger Dillmann,et al.  Teaching and learning of robot tasks via observation of human performance , 2004, Robotics Auton. Syst..

[15]  Aude Billard,et al.  Learning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators , 2011, Int. J. Robotics Res..

[16]  Kenji Suzuki,et al.  Coaching robot behavior using continuous physiological affective feedback , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[17]  Jun Morimoto,et al.  Learning parametric dynamic movement primitives from multiple demonstrations , 2011, Neural Networks.

[18]  Darwin G. Caldwell,et al.  Learning and Reproduction of Gestures by Imitation , 2010, IEEE Robotics & Automation Magazine.

[19]  Andrej Gams,et al.  On-line frequency adaptation and movement imitation for rhythmic robotic tasks , 2011, Int. J. Robotics Res..

[20]  Neville Hogan,et al.  Impedance control - An approach to manipulation. I - Theory. II - Implementation. III - Applications , 1985 .

[21]  Andrej Gams,et al.  Learning and adaptation of periodic motion primitives based on force feedback and human coaching interaction , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[22]  Ales Ude,et al.  Action sequencing using dynamic movement primitives , 2011, Robotica.

[23]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[24]  Thanh Vũ Thị Cẩm.  Science, Technology and Innovation , 2017, PAM 2017.

[25]  Stefan Schaal,et al.  Online movement adaptation based on previous sensor experiences , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Luigi Villani,et al.  Force Control , 2016, Springer Handbook of Robotics, 2nd Ed..

[27]  Minija Tamosiunaite,et al.  Joining Movement Sequences: Modified Dynamic Movement Primitives for Robotics Applications Exemplified on Handwriting , 2012, IEEE Transactions on Robotics.

[28]  Stefan Schaal,et al.  Learning force control policies for compliant manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Mitsuo Kawato,et al.  A via-point time optimization algorithm for complex sequential trajectory formation , 2004, Neural Networks.

[30]  Gordon Cheng,et al.  Coaching: An Approach to Efficiently and Intuitively Create Humanoid Robot Behaviors , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[31]  Dongheui Lee,et al.  Incremental kinesthetic teaching of motion primitives using the motion refinement tube , 2011, Auton. Robots.

[32]  Eric L. Sauser,et al.  An Approach Based on Hidden Markov Model and Gaussian Mixture Regression , 2010 .

[33]  Stefan Schaal,et al.  Towards Associative Skill Memories , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[34]  Carme Torras,et al.  Force-based robot learning of pouring skills using parametric hidden Markov models , 2013, 9th International Workshop on Robot Motion and Control.

[35]  Andrew W. Moore,et al.  Locally Weighted Learning , 1997, Artificial Intelligence Review.

[36]  Tamim Asfour,et al.  Particle Filter-based Fingertip Tracking with Circular Hough Transform Features , 2011, MVA.

[37]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[38]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[39]  Aude Billard,et al.  Incremental learning of gestures by imitation in a humanoid robot , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[40]  Andrej Gams,et al.  Coupling Movement Primitives: Interaction With the Environment and Bimanual Tasks , 2014, IEEE Transactions on Robotics.

[41]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[42]  Tamim Asfour,et al.  Learn to wipe: A case study of structural bootstrapping from sensorimotor experience , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[43]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[44]  Jun Morimoto,et al.  CB: A Humanoid Research Platform for Exploring NeuroScience , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[45]  Minija Tamosiunaite,et al.  Interaction learning for dynamic movement primitives used in cooperative robotic tasks , 2013, Robotics Auton. Syst..

[46]  Andrej Gams,et al.  On-line periodic movement and force-profile learning for adaptation to new surfaces , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[47]  Andrej Gams,et al.  On-line learning and modulation of periodic movements with nonlinear dynamical systems , 2009, Auton. Robots.

[48]  Jan Peters,et al.  Nonamemanuscript No. (will be inserted by the editor) Reinforcement Learning to Adjust Parametrized Motor Primitives to , 2011 .

[49]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[50]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation: Part I—Theory , 1985 .

[51]  Jimmy A. Jørgensen,et al.  Adaptation of manipulation skills in physical contact with the environment to reference force profiles , 2015, Auton. Robots.