Complex Sequential Tasks Learning with Bayesian Inference and Gaussian Mixture Model

Transferring skills to robots by demonstrations has been extensively researched for decades. However, the majority of the work focuses on individual or low-level task learning. Theories and applications for learning complex sequential tasks are not well-investigated. For this reason, this paper presents a unified top-down framework for complex tasks learning. Specifically, we conclude two critical objectives. First, a segmentation algorithm which can segment unstructured demonstrations into movement primitives (MPs) with minimal prior knowledge requirements needs to be proposed. Second, choosing a representation model used to jointly extract tasks constraints from the discovered MPs. To achieve the first goal, a change-point detection algorithm based on Bayesian inference is used. It can segment unstructured demonstrations online. Then, we propose to model MPs with dynamical system approximated by the Gaussian mixture models (GMMs), which is flexible and powerful in movement representation. Finally, the whole framework is evaluated by an open-and-place task on a real robot. Experiments show the segmentation accuracy can reach to 95.6% and the task can be replayed in new contexts successfully.

[1]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[2]  Sylvain Calinon,et al.  A tutorial on task-parameterized movement learning and retrieval , 2015, Intelligent Service Robotics.

[3]  Aude Billard,et al.  LEARNING MOTOR SKILLS BY IMITATION: A BIOLOGICALLY INSPIRED ROBOTIC MODEL , 2001, Cybern. Syst..

[4]  Aude Billard,et al.  Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models , 2011, IEEE Transactions on Robotics.

[5]  Scott Kuindersma,et al.  Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories , 2010, NIPS.

[6]  Michael I. Jordan,et al.  Sharing Features among Dynamical Systems with Beta Processes , 2009, NIPS.

[7]  Cory D. Kidd,et al.  HUMANOID ROBOTS AS COOPERATIVE PARTNERS FOR PEOPLE , 2004 .

[8]  Jan Peters,et al.  Learning movement primitive attractor goals and sequential skills from kinesthetic demonstrations , 2015, Robotics Auton. Syst..

[9]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[10]  Aude Billard,et al.  Learning from Humans , 2016, Springer Handbook of Robotics, 2nd Ed..

[11]  Rainer Palm,et al.  Programming by Demonstration of Pick-and-Place Tasks for Industrial Manipulators using Task Primitives , 2007, 2007 International Symposium on Computational Intelligence in Robotics and Automation.

[12]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[13]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[14]  Oliver Kroemer,et al.  Towards Robot Skill Learning: From Simple Skills to Table Tennis , 2013, ECML/PKDD.

[15]  Dmitry Berenson,et al.  Simultaneous learning of hierarchy and primitives for complex robot tasks , 2019, Auton. Robots.

[16]  Daniel H. Grollman,et al.  Incremental learning of subtasks from unsegmented demonstration , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  A. Billard,et al.  Learning the Nonlinear Multivariate Dynamics of Motion of Robotic Manipulators , 2009 .

[18]  Douglas R. Smith,et al.  The Design of Divide and Conquer Algorithms , 1985, Sci. Comput. Program..

[19]  Aude Billard,et al.  Learning to Play Minigolf: A Dynamical System-Based Approach , 2012, Adv. Robotics.

[20]  Dana Kulic,et al.  Incremental learning of human behaviors using hierarchical hidden Markov models , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Scott Niekum,et al.  Learning grounded finite-state representations from unstructured demonstrations , 2015, Int. J. Robotics Res..

[22]  Aude Billard,et al.  A dynamical system approach to realtime obstacle avoidance , 2012, Autonomous Robots.

[23]  Steven Lemm,et al.  A Dynamic HMM for On-line Segmentation of Sequential Data , 2001, NIPS.

[24]  Jan Peters,et al.  Learning movement primitive libraries through probabilistic segmentation , 2017, Int. J. Robotics Res..

[25]  Sang Hyoung Lee,et al.  Autonomous framework for segmenting robot trajectories of manipulation task , 2015, Auton. Robots.

[26]  Chrystopher L. Nehaniv,et al.  Imitation as a Dual-Route Process Featuring Predictive and Learning Components: A Biologically Plausible Computational Model , 2002 .

[27]  Ales Ude,et al.  Door opening by joining reinforcement learning and intelligent control , 2017, 2017 18th International Conference on Advanced Robotics (ICAR).

[28]  Elsa Andrea Kirchner,et al.  Velocity-Based Multiple Change-Point Inference for Unsupervised Segmentation of Human Movement Behavior , 2014, 2014 22nd International Conference on Pattern Recognition.

[29]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[30]  Gillian M. Hayes,et al.  Imitation as a dual-route process featuring prediction and learning components: A biologically plaus , 2002 .

[31]  Scott Niekum,et al.  Learning and generalization of complex tasks from unstructured demonstrations , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Jonathan Feng-Shun Lin,et al.  Online Segmentation of Human Motion for Automated Rehabilitation Exercise Analysis , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Dana Kulic,et al.  Online Segmentation and Clustering From Continuous Observation of Whole Body Motions , 2009, IEEE Transactions on Robotics.

[34]  Ioannis A. Kakadiaris,et al.  A Review of Human Activity Recognition Methods , 2015, Front. Robot. AI.

[35]  Jan Peters,et al.  Probabilistic Movement Primitives , 2013, NIPS.

[36]  M. Matarić,et al.  Evaluation Metrics and Results of Human Arm Movement Imitation , 2000 .

[37]  Jan Peters,et al.  Probabilistic segmentation applied to an assembly task , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[38]  Stefan Schaal,et al.  Movement segmentation using a primitive library , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.