Controlling bots in a First Person Shooter game using genetic algorithms

In this paper we employ a steady state genetic algorithm to evolve different types of behaviour for bots in the Unreal Tournament 2004™ computer game. For this purpose we define three fitness functions which are based on the number of enemies killed, the lifespan of the bot and a combination of both. Long run experiments were carried out, in which the evolved bots' behaviours outperform those of standard bots supplied by the game, particularly in those cases where the fitness involves a measure of the bot's lifespan. Also, there is an increase in the number of items collected, and the behaviours tend to become less aggressive, tending instead towards a more optimised combat style. Further “short run” experiments were carried out with a further type of fitness function defined, based on the number of items picked. In these cases the bots evolve performances towards the goal they have been aimed, with no other behaviours arising, except in the case of the multiple objective one. We conclude that in order to evolve interesting behaviours more complex fitness functions are needed, and not necessarily ones that directly include the goal we are aiming for.

[1]  Taylor L. Booth,et al.  Sequential machines and automata theory , 1967 .

[2]  D. C. Cooper,et al.  Sequential Machines and Automata Theory , 1968, Comput. J..

[3]  J. D. Schaffer,et al.  Real-Coded Genetic Algorithms and Interval-Schemata , 1992, FOGA.

[4]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[5]  Sushil J. Louis,et al.  Using a genetic algorithm to tune first-person shooter bots , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[6]  Controlling Unreal Tournament 2004 Bots with the Logic-based Action Language GOLOG , 2005, AIIDE.

[7]  Sung-Bae Cho,et al.  Evolving Reactive NPCs for the Real-Time Simulation Game , 2005, CIG.

[8]  Risto Miikkulainen,et al.  Integration and Evaluation of Exploration-Based Learning in Games , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[9]  Sung Hoon Jung,et al.  Exploiting Intelligence in Fighting Action Games Using Neural Networks , 2006, IEICE Trans. Inf. Syst..

[10]  Ruck Thawonmas,et al.  A method for online adaptation of computer-game AI rulebase , 2006, ACE '06.

[11]  Philip Hingston,et al.  Bots trained to play like a human are more fun , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[12]  Marcus Gallagher,et al.  Creating a multi-purpose first person shooter bot with reinforcement learning , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[13]  Risto Miikkulainen,et al.  Evolving multi-modal behavior in NPCs , 2009, CIG.

[14]  Julian Togelius,et al.  Hierarchical controller learning in a First-Person Shooter , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[15]  Clare Bates Congdon,et al.  Agent Smith: Towards an evolutionary rule-based agent for interactive dynamic games , 2009, 2009 IEEE Congress on Evolutionary Computation.