Learning Hierarchical Skills from Observation

This paper addresses the problem of learning control skills from observation. In particular, we show how to infer a hierarchical, reactive program that reproduces and explains the observed actions of other agents, specifically the elements that are shared across multiple individuals. We infer these programs using a three-stage process that learns flat unordered rules, combines these rules into a classification hierarchy, and finally translates this structure into a hierarchical reactive program. The resulting program is concise and easy to understand, making it possible to view program induction as a practical technique for knowledge acquisition.

[1]  Ross D. Shachter,et al.  Value-driven agents , 2001 .

[2]  Bruce A. Draper,et al.  Behavioral Cloning of Student Pilots with Modular Neural Networks , 2000, ICML.

[3]  Marcel Schoppers,et al.  Universal Plans for Reactive Robots in Unpredictable Environments , 1987, IJCAI.

[4]  Amy L. Lansky,et al.  A Procedural Logic , 1985, IJCAI.

[5]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[6]  Pat Langley,et al.  Separating Skills from Preference: Using Learning to Program by Reward , 2002, ICML.

[7]  Robert James Firby,et al.  Adaptive execution in complex dynamic worlds , 1989 .

[8]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  Ross D. Shachter,et al.  Using background knowledge to speed reinforcement learning in physical agents , 2001, AGENTS '01.

[11]  Claude Sammut,et al.  Automatic construction of reactive control systems using symbolic machine learning , 1996, The Knowledge Engineering Review.

[12]  Dean Pomerleau,et al.  Rapidly Adapting Artificial Neural Networks for Autonomous Navigation , 1990, NIPS.

[13]  Ivan Bratko,et al.  Reconstructing Human Skill with Machine Learning , 1994, ECAI.

[14]  金田 重郎,et al.  C4.5: Programs for Machine Learning (書評) , 1995 .

[15]  Nils J. Nilsson,et al.  Teleo-Reactive Programs for Agent Control , 1993, J. Artif. Intell. Res..

[16]  Tom M. Mitchell,et al.  LEAP: A Learning Apprentice for VLSI Design , 1985, IJCAI.

[17]  Henry Lieberman,et al.  Watch what I do: programming by demonstration , 1993 .

[18]  Pat Langley,et al.  Learning Context-Free Grammars with a Simplicity Bias , 2000, ECML.

[19]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[20]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .