Building behavior trees from observations in real-time strategy games

This paper presents a novel use of motif-finding techniques from computational biology to find recurring action sequences across many observations of expert humans carrying out a complex task. Information about recurring action sequences is used to produce a behavior tree without any additional domain information besides a simple similarity metric - no action models or reward functions are provided. This technique is applied to produce a behavior tree for strategic-level actions in the real-time strategy game StarCraft. The behavior tree was able to represent and summarise a large amount of information from the expert behavior examples much more compactly. The method could still be improved by discovering reactive actions present in the expert behavior and encoding these in the behavior tree.

[1]  Pedro Pablo Gómez-Martín,et al.  Combining Expert Knowledge and Learning from Demonstration in Real-Time Strategy Games , 2011, ICCBR.

[2]  Michael Buro,et al.  RTS Games and Real-Time AI Research , 2003 .

[3]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[4]  Arnav Jhala,et al.  Building Human-Level AI for Real-Time Strategy Games , 2011, AAAI Fall Symposium: Advances in Cognitive Systems.

[5]  Martin C. Frith,et al.  Discovering Sequence Motifs with Arbitrary Insertions and Deletions , 2008, PLoS Comput. Biol..

[6]  Xuemei Wang,et al.  Learning by Observation and Practice: An Incremental Approach for Planning Operator Acquisition , 1995, ICML.

[7]  Pedro Pablo Gómez-Martín,et al.  Extending Case-Based Planning with Behavior Trees , 2011, FLAIRS Conference.

[8]  Arnav Jhala,et al.  Reactive planning idioms for multi-scale game AI , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[9]  Daniel Borrajo.  Learning action durations from executions , 2007 .

[10]  Hector Muñoz-Avila,et al.  Learning Hierarchical Task Networks for Nondeterministic Planning Domains , 2009, IJCAI.

[11]  Leslie Pack Kaelbling,et al.  Learning Probabilistic Relational Planning Rules , 2004, ICAPS.

[12]  Santiago Ontañón,et al.  Case Acquisition Strategies for Case-Based Reasoning in Real-Time Strategy Games , 2012, FLAIRS.

[13]  Glen Robertson,et al.  A Review of Real-Time Strategy Game AI , 2014, AI Mag..

[14]  Hector Muñoz-Avila,et al.  Learning HTN Method Preconditions and Action Models from Partial Observations , 2009, IJCAI.

[15]  Dafna Shahaf,et al.  Learning Partially Observable Action Schemas , 2006, AAAI.

[16]  Pedro Pablo Gómez-Martín,et al.  Query-Enabled Behavior Trees , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[17]  Paul R. Cohen,et al.  Learning Planning Operators in Real-World, Partially Observable Environments , 2000, AIPS.

[18]  Prasad Tadepalli,et al.  Hierarchical structure discovery and transfer in sequential decision problems , 2011 .

[19]  David W. Aha,et al.  LEARNING PRECONDITIONS FOR PLANNING FROM PLAN TRACES AND HTN STRUCTURE , 2005, Comput. Intell..

[20]  John E. Laird,et al.  Learning Goal-Oriented Hierarchical Tasks from Situated Interactive Instruction , 2014, AAAI.

[21]  Manuela M. Veloso,et al.  DISTILL: Learning Domain-Specific Planners by Example , 2003, ICML.

[22]  Simon Colton,et al.  Evolving Behaviour Trees for the Commercial Game DEFCON , 2010, EvoApplications.

[23]  Rudolf Kadlec,et al.  Evoluce chovan´õ inteligentn´õch agentû u v poÿc´õtaÿcovych hrach Evolution of intelligent agent behaviour in computer games , 2008 .

[24]  Arnav Jhala,et al.  Sequential Pattern Mining in StarCraft: Brood War for Short and Long-Term Goals , 2014 .

[25]  Ian D. Watson,et al.  Integrating Case-Based Reasoning with Reinforcement Learning for Real-Time Strategy Game Micromanagement , 2014, PRICAI.

[26]  Glen Robertson,et al.  An Improved Dataset and Extraction Process for Starcraft AI , 2014, FLAIRS Conference.

[27]  Santiago Ontañón,et al.  A Survey of Real-Time Strategy Game AI Research and Competition in StarCraft , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[28]  Pat Langley,et al.  Learning hierarchical task networks by observation , 2006, ICML.

[29]  Paolo Traverso,et al.  Automated Planning: Theory & Practice , 2004 .

[30]  Thomas G. Dietterich,et al.  Learning Probabilistic Behavior Models in Real-Time Strategy Games , 2011, AIIDE.

[31]  Pierre Bessière,et al.  A Bayesian Model for Plan Recognition in RTS Games Applied to StarCraft , 2011, AIIDE.

[32]  Michael Buro,et al.  Real-Time Strategy Game Competitions , 2012, AI Mag..

[33]  Qiang Yang,et al.  Learning Actions Models from Plan Examples with Incomplete Knowledge , 2005, ICAPS.

[34]  Qiang Yang,et al.  Learning action models from plan examples using weighted MAX-SAT , 2007, Artif. Intell..

[35]  Hector Muñoz-Avila,et al.  HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required , 2008, AAAI.