Real-Time Heuristic Search for Pathfinding in Video Games

Game pathfinding is a challenging problem due to a limited amount of per-frame CPU time commonly shared among many simultaneously pathfinding agents. The challenge is rising with each new generation of games due to progressively larger and more complex environments and larger numbers of agents pathfinding in them. Algorithms based on A* tend to scale poorly as they must compute a complete, possibly abstract, path for each agent before the agent can move. Real-time heuristic search algorithms satisfy a constant bound on the amount of planning per move, independent of problem size. These algorithms are thus a promising approach to large scale multi-agent pathfinding in video games. However, until recently, real-time heuristic search algorithms universally exhibited a visually unappealing “scrubbing” behavior by repeatedly revisiting map locations. This had prevented their adoption by video game developers. In this chapter, we review three modern search algorithms which address the “scrubbing” problem in different ways. Each algorithm presentation is complete with an empirical evaluation on game maps.

[1]  Nathan R. Sturtevant,et al.  Memory-Efficient Abstractions for Pathfinding , 2007, AIIDE.

[2]  Nathan R. Sturtevant,et al.  On learning in agent-centered search , 2010, AAMAS.

[3]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[4]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[5]  Mitja Luštrek,et al.  PATHOLOGY IN SINGLE-AGENT SEARCH , 2005 .

[6]  Simon Parsons,et al.  Do the right thing - studies in limited rationality by Stuart Russell and Eric Wefald, MIT Press, Cambridge, MA, £24.75, ISBN 0-262-18144-4 , 1994, The Knowledge Engineering Review.

[7]  Vadim Bulitko,et al.  Lookahead Pathology in Real-Time Path-Finding , 2006, AAAI.

[8]  Vadim Bulitko,et al.  Lookahead Pathologies and Meta-level Control in Real-Time Heuristic Search , 2003 .

[9]  Toru Ishida,et al.  Moving Target Search with Intelligence , 1992, AAAI.

[10]  Sven Koenig,et al.  Real-time adaptive A* , 2006, AAMAS '06.

[11]  Blai Bonet,et al.  Planning as heuristic search , 2001, Artif. Intell..

[12]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[13]  Blai Bonet,et al.  Learning Depth-First Search: A Unified Approach to Heuristic Search in Deterministic and Non-Deterministic Settings, and Its Application to MDPs , 2006, ICAPS.

[14]  Stuart J. Russell,et al.  Do the right thing - studies in limited rationality , 1991 .

[15]  Nathan R. Sturtevant,et al.  Partial Pathfinding Using Map Abstraction and Refinement , 2005, AAAI.

[16]  Yngvi Björnsson,et al.  Value BackPropagation versus Backtracking in Real-Time Heuristic Search , 2006 .

[17]  Vadim Bulitko,et al.  Learning for Adaptive Real-time Search , 2004, ArXiv.

[18]  Vadim Bulitko,et al.  Lookahead Pathologies for Single Agent Search , 2003, IJCAI.

[19]  Nathan R. Sturtevant,et al.  TBA*: Time-Bounded A* , 2009, IJCAI.

[20]  Sven Koenig,et al.  A comparison of fast search methods for real-time situated agents , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[21]  Vadim Bulitko,et al.  kNN LRTA*: Simple Subgoaling for Real-Time Search , 2009, AIIDE.

[22]  Blai Bonet,et al.  A Robust and Fast Action Selection Mechanism for Planning , 1997, AAAI/IAAI.

[23]  Pedro Meseguer,et al.  LRTA*(k) , 2005, IJCAI.

[24]  Reza Zamani,et al.  An Admissible Heuristic Search Algorithm , 1993, ISMIS.

[25]  Jonathan Schaeffer,et al.  Dynamic Control in Real-Time Heuristic Search , 2008, J. Artif. Intell. Res..

[26]  Toru Ishida,et al.  Controlling the learning process of real-time heuristic search , 2003, Artif. Intell..

[27]  V. Bulitko,et al.  Learning in Real-Time Search: A Unifying Framework , 2011, J. Artif. Intell. Res..

[28]  Improving Convergence of LRTA * ( k ) ∗ , 2005 .

[29]  Sebastian Thrun,et al.  ARA*: Anytime A* with Provable Bounds on Sub-Optimality , 2003, NIPS.

[30]  Richard E. Korf,et al.  Depth-First Iterative-Deepening: An Optimal Admissible Tree Search , 1985, Artif. Intell..

[31]  David Furcy,et al.  Heuristic Search-Based Replanning , 2002, AIPS.

[32]  David Furcy,et al.  Speeding up the Convergence of Real-Time Search , 2000, AAAI/IAAI.

[33]  Nathan R. Sturtevant,et al.  Graph Abstraction in Real-time Heuristic Search , 2007, J. Artif. Intell. Res..

[34]  Vadim Bulitko,et al.  Real-Time Heuristic Search with a Priority Queue , 2007, IJCAI.

[35]  Sebastian Thrun,et al.  Anytime Dynamic A*: An Anytime, Replanning Algorithm , 2005, ICAPS.

[36]  Jorg Homann.  A Heuristic for Domain Independent Planning and Its Use in an Enforced Hill-Climbing Algorithm , 2000 .

[37]  Vadim Bulitko,et al.  Case-Based Subgoaling in Real-Time Heuristic Search for Video Game Pathfinding , 2010, J. Artif. Intell. Res..

[38]  Anthony Stentz,et al.  The Focussed D* Algorithm for Real-Time Replanning , 1995, IJCAI.