Incremental learning of subtasks from unsegmented demonstration

We propose to incrementally learn the segmentation of a demonstrated task into subtasks and the individual subtask policies themselves simultaneously. Previous robot learning from demonstration techniques have either learned the individual subtasks in isolation, combined known subtasks, or used knowledge of the overall task structure to perform segmentation. Our infinite mixture of experts approach instead automatically infers an appropriate partitioning (number of subtasks and assignment of data points to each one) directly from the data. We illustrate the applicability of our technique by learning a suitable set of subtasks from the demonstration of a finite-state machine robot soccer goal scorer.

[1]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[2]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[3]  Tetsunari Inamura Masayuki Inaba Hirochika.  Acquisition of Probabilistic Behavior Decision Model based on the Interactive Teaching Method , 2001 .

[4]  L. Csató.  Gaussian processes:iterative sparse approximations , 2002 .

[5]  Stefan Schaal,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[6]  Paul Fearnhead,et al.  Particle filters for mixture models with an unknown number of components , 2004, Stat. Comput..

[7]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[8]  Dana H. Ballard,et al.  Learning to perceive and act by trial and error , 1991, Machine Learning.

[9]  Simon Osindero,et al.  An Alternative Infinite Mixture Of Gaussian Process Experts , 2005, NIPS.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  J. Pitman.  Combinatorial Stochastic Processes , 2006 .

[12]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[13]  Daniel H. Grollman,et al.  Sparse incremental learning for interactive robot control policy estimation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[14]  Eric Fritzinger,et al.  Learning behavior fusion from demonstration , 2008 .

[15]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[16]  Sethu Vijayakumar,et al.  Methods for Learning Control Policies from Variable-Constraint Demonstrations , 2010, From Motor Learning to Interaction Learning in Robots.

[17]  Daniel H. Grollman,et al.  Can We Learn Finite State Machine Robot Controllers from Interactive Demonstration? , 2010, From Motor Learning to Interaction Learning in Robots.