Search in Real-Time Video Games

This chapter arises from the discussions of an experienced international group of researchers interested in the potential for creative application of algorithms for searching finite discrete graphs, which have been highly successful in a wide range of application areas, to address a broad range of problems arising in video games. The chapter first summarises the state of the art in search algorithms for games. It then considers the challenges in implementing these algorithms in video games (particularly real time strategy and first-person games) and ways of creating searchable discrete representations of video game decisions (for example as state-action graphs). Finally the chapter looks forward to promising techniques which might bring some of the success achieved in games such as Go and Chess, to real-time video games. For simplicity, we will consider primarily the objective of maximising playing strength, and consider games where this is a challenging task, which results in interesting gameplay.

[1]  Peter I. Cowling,et al.  Monte Carlo Tree Search with macro-actions and heuristic route planning for the Physical Travelling Salesman Problem , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[2]  Richard E. Korf,et al.  Depth-First Iterative-Deepening: An Optimal Admissible Tree Search , 1985, Artif. Intell..

[3]  Yngvi Björnsson,et al.  CadiaPlayer: A Simulation-Based General Game Player , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[4]  Andrew G. Barto,et al.  Using relative novelty to identify useful temporal abstractions in reinforcement learning , 2004, ICML.

[5]  Steve Rabin,et al.  AI Game Programming Wisdom , 2002 .

[6]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[7]  David J. Slate,et al.  Chess 4.5-The Northwestern University chess program , 1988 .

[8]  Austin Tate,et al.  O-Plan: The open Planning Architecture , 1991, Artif. Intell..

[9]  Nathan R. Sturtevant,et al.  Prob-Maxn: Playing N-Player Games with Opponent Models , 2006, AAAI.

[10]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[11]  Michael L. Littman,et al.  Sample-Based Planning for Continuous Action Markov Decision Processes , 2011, ICAPS.

[12]  Hector Muñoz-Avila,et al.  Hierarchical Plan Representations for Encoding Strategic Game AI , 2005, AIIDE.

[13]  Bruno Bouzy,et al.  Monte-Carlo strategies for computer Go , 2006 .

[14]  Keki B. Irani,et al.  An Algorithmic Solution of N-Person Games , 1986, AAAI.

[15]  Martin J. Osborne,et al.  An Introduction to Game Theory , 2003 .

[16]  Phillip Gowlett,et al.  Moving Forward with Computational Red Teaming , 2011 .

[17]  Rudolf Kadlec,et al.  Evoluce chovan´õ inteligentn´õch agentû u v poÿc´õtaÿcovych hrach Evolution of intelligent agent behaviour in computer games , 2008 .

[18]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[19]  Julian Togelius,et al.  Search-Based Procedural Content Generation: A Taxonomy and Survey , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[20]  Andrew G. Barto,et al.  Efficient skill learning using abstraction selection , 2009, IJCAI 2009.

[21]  Dana S. Nau,et al.  SHOP2: An HTN Planning System , 2003, J. Artif. Intell. Res..

[22]  Michael Buro,et al.  RTS Games and Real-Time AI Research , 2003 .

[23]  Daniel M. Wolpert,et al.  Hierarchical MOSAIC for movement generation , 2003 .

[24]  Nathan R. Sturtevant,et al.  TBA*: Time-Bounded A* , 2009, IJCAI.

[25]  Dana S. Nau,et al.  Forward-Chaining Planning in Nondeterministic Domains , 2004, AAAI.

[26]  Sushil J. Louis,et al.  Playing to learn: case-injected genetic algorithms for learning to play computer games , 2005, IEEE Transactions on Evolutionary Computation.

[27]  Jonathan Schaeffer,et al.  Checkers Is Solved , 2007, Science.

[28]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[29]  Jonathan Schaeffer,et al.  The History Heuristic and Alpha-Beta Search Enhancements in Practice , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Donald E. Knuth,et al.  An Analysis of Alpha-Beta Pruning , 1975, Artif. Intell..

[31]  Hector Muñoz-Avila,et al.  HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required , 2008, AAAI.

[32]  Marco Pistore,et al.  Task decomposition on abstract states, for planning under nondeterminism , 2009, Artif. Intell..

[33]  Michael Mateas,et al.  Time in Video Games: A Survey and Analysis , 2010 .

[34]  Dana S. Nau,et al.  Using Domain-Configurable Search Control for Probabilistic Planning , 2005, AAAI.

[35]  Mark H. M. Winands,et al.  Enhancements for Multi-Player Monte-Carlo Tree Search , 2010, Computers and Games.

[36]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[37]  Bruno Bouzy,et al.  Associating domain-dependent knowledge and Monte Carlo approaches within a Go program , 2005, Inf. Sci..

[38]  Peter I. Cowling,et al.  Information Set Monte Carlo Tree Search , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[39]  Murray Campbell,et al.  Singular Extensions: Adding Selectivity to Brute-Force Searching , 1990, Artif. Intell..

[40]  Martijn Schut,et al.  A hierarchically-layered multiplayer bot system for a first-person shooter , 2007 .

[41]  Nathan R. Sturtevant,et al.  Partial Pathfinding Using Map Abstraction and Refinement , 2005, AAAI.

[42]  Csaba Szepesvári,et al.  Online Optimization in X-Armed Bandits , 2008, NIPS.

[43]  Richard E. Korf,et al.  On Pruning Techniques for Multi-Player Games , 2000, AAAI/IAAI.

[44]  Ross Graham,et al.  Pathfinding in Computer Games , 2003 .

[45]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[46]  Judea Pearl,et al.  Asymptotic Properties of Minimax Trees and Game-Searching Procedures , 1980, Artif. Intell..

[47]  Tomaso Poggio,et al.  Generalization in vision and motor control , 2004, Nature.

[48]  Andrew G. Barto,et al.  Causal Graph Based Decomposition of Factored MDPs , 2006, J. Mach. Learn. Res..

[49]  Hector Muñoz-Avila,et al.  RETALIATE: Learning Winning Policies in First-Person Shooter Games , 2007, AAAI.

[50]  David Silver,et al.  Monte-Carlo tree search and rapid action value estimation in computer Go , 2011, Artif. Intell..

[51]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[52]  H. J. van den Herik,et al.  Opponent Modelling and Commercial Games , 2005 .

[53]  Ira Pohl,et al.  Heuristic Search Viewed as Path Finding in a Graph , 1970, Artif. Intell..

[54]  Nathan R. Sturtevant,et al.  An Analysis of UCT in Multi-Player Games , 2008, J. Int. Comput. Games Assoc..

[55]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[56]  Inon Zuckerman,et al.  Modeling social preferences in multi-player games , 2011, AAMAS.

[57]  Jonathan Schaeffer,et al.  Best-First Fixed-Depth Minimax Algorithms , 1996, J. Int. Comput. Games Assoc..

[58]  Dana S. Nau,et al.  Computer Bridge - A Big Win for AI Planning , 1998, AI Mag..

[59]  Michael Buro,et al.  Efficient Triangulation-Based Pathfinding , 2006, AAAI.

[60]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[61]  Adi Botea,et al.  Near Optimal Hierarchical Path-Finding , 2004, J. Game Dev..

[62]  David E. Wilkins,et al.  Practical planning - extending the classical AI planning paradigm , 1989, Morgan Kaufmann series in representation and reasoning.

[63]  Albert L. Zobrist,et al.  A New Hashing Method with Application for Game Playing , 1990 .

[64]  Jonathan Schaeffer,et al.  Block A*: Database-Driven Search with Applications in Any-Angle Path-Planning , 2011, AAAI.

[65]  A. V. Uskov,et al.  Programming a computer to play chess , 1970 .

[66]  Richard E. Korf.  Multi-Player Alpha-Beta Pruning , 1991, Artif. Intell..

[67]  Austin Tate,et al.  O-Plan2: an Open Architecture for Command, Planning and Control , 2006 .

[68]  Adi Botea,et al.  Hierarchical path planning for multi-size agents in heterogeneous environments , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[69]  Philip Hingston,et al.  Using Monte Carlo Tree Search for replanning in a multistage simultaneous game , 2012, 2012 IEEE Congress on Evolutionary Computation.

[70]  Sushil J. Louis,et al.  Towards the Co-Evolution of Influence Map Tree Based Strategy Game Players , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[71]  Michael Buro,et al.  Adversarial Planning Through Strategy Simulation , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[72]  Martin V. Butz,et al.  Anticipations, Brains, Individual and Social Behavior: An Introduction to Anticipatory Systems , 2007, SAB ABiALS.

[73]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents , 2012, J. Artif. Intell. Res..

[74]  S.M. Lucas,et al.  Evolutionary computation and games , 2006, IEEE Computational Intelligence Magazine.

[75]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.