Learning to be a Bot: Reinforcement Learning in Shooter Games

This paper demonstrates the applicability of reinforcement learning for first person shooter bot artificial intelligence. Reinforcement learning is a machine learning technique where an agent learns a problem through interaction with the environment. The Sarsa(λ) algorithm will be applied to a first person shooter bot controller to learn the tasks of (1) navigation and item collection, and (2) combat. The results will show the validity and diversity of reinforcement learning in a first person shooter environment.

[1]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[2]  Gerald Tesauro,et al.  Temporal difference learning and TD-Gammon , 1995, CACM.

[3]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[4]  Se-Young Oh,et al.  TD based reinforcement learning using neural networks in control problems with continuous action space , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  Manuela M. Veloso,et al.  Rational and Convergent Learning in Stochastic Games , 2001, IJCAI.

[6]  Manuela M. Veloso,et al.  Multiagent learning using a variable learning rate , 2002, Artif. Intell..

[7]  Bruce Blumberg,et al.  Integrated learning for interactive synthetic characters , 2002, SIGGRAPH.

[8]  Daniel Sanchez-Crespo Dalmau,et al.  Core Techniques and Algorithms in Game Programming , 2003 .

[9]  I. Sprinkhuizen-Kuyper,et al.  Online Adaptation of Computer Game Opponent AI , 2003 .

[10]  Jason Jones.  Benefits of Genetic Algorithms in Simulations for Game Designers , 2003 .

[11]  Eric O. Postma,et al.  On-line Adaptation of Game Opponent AI with Dynamic Scripting , 2004, Int. J. Intell. Games Simul..

[12]  Thore Graepel,et al.  LEARNING TO FIGHT , 2004 .

[13]  David M. Bourg,et al.  AI for Game Developers , 2004 .

[14]  John E. Laird,et al.  Soar-RL: integrating reinforcement learning with Soar , 2005, Cognitive Systems Research.

[15]  Ah-Hwee Tan,et al.  Self-organizing cognitive agents and reinforcement learning in multi-agent environment , 2005, IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[16]  S. Levy,et al.  Evolving AI Opponents in a First-Person-Shooter Video Game , 2005, AAAI.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Gillian Hayes,et al.  Group utility functions: learning equilibria between groups of agents in computer games by modifying the reinforcement signal , 2005, 2005 IEEE Congress on Evolutionary Computation.

[19]  Jonathan Schaeffer,et al.  Generating Ambient Behaviors in Computer Role-Playing Games , 2005, IEEE Intelligent Systems.

[20]  Kathryn E. Merrick,et al.  Motivated reinforcement learning for non-player characters in persistent computer game worlds , 2006, ACE '06.

[21]  Eric O. Postma,et al.  Adaptive game AI with dynamic scripting , 2006, Machine Learning.

[22]  H. Jaap van den Herik,et al.  Automatic Rule Ordering for Dynamic Scripting , 2007, AIIDE.

[23]  Hector Muñoz-Avila,et al.  RETALIATE: Learning Winning Policies in First-Person Shooter Games , 2007, AAAI.

[24]  Richard S. Sutton,et al.  Reinforcement Learning of Local Shape in the Game of Go , 2007, IJCAI.

[25]  Duane Szafron,et al.  A Demonstration of Agent Learning with Action-Dependent Learning Rates in Computer Role-Playing Games , 2008, AIIDE.