Learning Hierarchical Skills for Game Agents from Video of Human Behavior

Abstract : Developing autonomous agents for computer games is often a lengthy and expensive undertaking that requires manual encoding of detailed and complex knowledge. In this paper we show how to acquire hierarchical skills for controlling a team of simulated football players by observing video of college football play. We then demonstrate the results in the Rush 2008 football simulator, showing that the learned skills have high fidelity with respect to the observed video and are robust to changes in the environment. Finally, we conclude with discussions of this work and of possible improvements.

[1]  Julian N. Marewski,et al.  Proceedings of the 31st Annual Meeting of the Cognitive Science Society , 2009 .

[2]  John E. Laird,et al.  EXAMPLE-DRIVEN DIAGRAMMATIC TOOLS FOR RAPID KNOWLEDGE ACQUISITION , 2003 .

[3]  John E. Laird,et al.  Soar-RL: integrating reinforcement learning with Soar , 2005, Cognitive Systems Research.

[4]  Luc De Raedt,et al.  Proceedings of the 22nd international conference on Machine learning , 2005 .

[5]  Adi Botea,et al.  Offline Planning with Hierarchical Task Networks in Video Games , 2007, AIIDE.

[6]  Alan Fern,et al.  Mixture-of-Parts Pictorial Structures for Objects with Variable Part Sets , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  Alan Fern,et al.  Discriminatively trained particle filters for complex multi-object tracking , 2009, CVPR.

[8]  Hector Muñoz-Avila,et al.  HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required , 2008, AAAI.

[9]  Kenneth D. Forbus,et al.  Companion Cognitive Systems: A Step towards Human-Level AI , 2004, AI Mag..

[10]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[11]  Gillian Hayes,et al.  Group utility functions: learning equilibria between groups of agents in computer games by modifying the reinforcement signal , 2005, 2005 IEEE Congress on Evolutionary Computation.

[12]  Pat Langley,et al.  Learning hierarchical task networks by observation , 2006, ICML.

[13]  Allen Newell,et al.  Chunking in Soar: The anatomy of a general learning mechanism , 1985, Machine Learning.

[14]  Verzekeren Naar Sparen,et al.  Cambridge , 1969, Humphrey Burton: In My Own Time.

[15]  David J. Stracuzzi,et al.  Representing and Reasoning over Time in a Symbolic Cognitive Architecture , 2009 .