Curriculum Learning for Motor Skills

Humans and animals acquire their wide repertoire of motor skills through an incremental learning process, during which progressively more complex skills are acquired and subsequently integrated with prior abilities. Inspired by this general idea, we develop an approach for learning motor skills based on a two-level curriculum. At the high level, the curriculum specifies an order in which different skills should be learned. At the low level, the curriculum defines a process for learning within a skill. We develop a set of integrated motor skills for a planar articulated figure capable of doing parameterized hops, flips, rolls, and acrobatic sequences. The same curriculum can be applied to yield individualized motor skill sets for articulated figures of varying proportions.

[1]  Timothy D. Lee,et al.  Motor Control and Learning: A Behavioral Emphasis , 1982 .

[2]  R. Murray,et al.  Nonlinear controllers for non-integrable systems: the Acrobot example , 1990, 1990 American Control Conference.

[3]  Mark W. Spong,et al.  The swing up control problem for the Acrobot , 1995 .

[4]  Gary Boone,et al.  Minimum-time control of the Acrobot , 1997, Proceedings of International Conference on Robotics and Automation.

[5]  Andrew G. Barto,et al.  PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning , 2002, ICML.

[6]  Andrew Y. Ng,et al.  Shaping and policy search in reinforcement learning , 2003 .

[7]  Minoru Asada,et al.  Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning , 2005, Machine Learning.

[8]  Philippe Beaudoin,et al.  Continuation methods for adapting simulated skills , 2008, ACM Trans. Graph..

[9]  Stephen Hart,et al.  The development of hierarchical knowledge in robot systems , 2009 .

[10]  Matthew E. Taylor,et al.  Abstraction and Generalization in Reinforcement Learning: A Summary and Framework , 2009, ALA.

[11]  Jan Peters,et al.  Learning complex motions by sequencing simpler motion templates , 2009, ICML '09.

[12]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[13]  Andrew G. Barto,et al.  Skill Discovery in Continuous Reinforcement Learning Domains using Skill Chaining , 2009, NIPS.

[14]  Andrew G. Barto,et al.  Competence progress intrinsic motivation , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[15]  Matthew E. Taylor,et al.  Adaptive and Learning Agents, Second Workshop, ALA 2009, Held as Part of the AAMAS 2009 Conference in Budapest, Hungary, May 12, 2009, Revised Selected Papers , 2010, ALA.