Fitness Biasing for evolving an Xpilot combat agent

In this paper we present an application of Fitness Biasing, a type of Punctuated Anytime Learning, for learning autonomous agents in the space combat game Xpilot. Fitness Biasing was originally developed as a means of linking the model to the actual robot in evolutionary robotics. We use fitness biasing with a standard genetic algorithm to learn control programs for a video game agent in real-time. Xpilot-AI, an Xpilot add-on designed for testing learning systems, is used to evolve the controller in the background while periodic checks in normal game play are used to compensate for errors produced by running the system at a high frame rate. The resultant learned controllers are comparable to our best hand-coded Xpilot-AI bots, display complex behavior that resemble human strategies, and are capable of adapting to a changing enemy in real-time.

[1]  Martin Allen,et al.  Real-time ai in xpilot using reinforcement learning , 2010, 2010 World Automation Congress.

[2]  Gary B. Parker,et al.  Evolving Parameters for Xpilot Combat Agents , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[3]  Gary B. Parker,et al.  Using evolution strategies for the real-time learning of controllers for autonomous agents in Xpilot-AI , 2010, IEEE Congress on Evolutionary Computation.

[4]  Gary B. Parker,et al.  Using a Queue Genetic Algorithm to Evolve Xpilot Control Strategies on a Distributed System , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[5]  Simon Colton,et al.  Combining AI Methods for Learning Bots in a Real-Time Strategy Game , 2009, Int. J. Comput. Games Technol..

[6]  Gary B. Parker,et al.  The Evolution of Multi-Layer Neural Networks for the Control of Xpilot Agents , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[7]  John J. Grefenstette,et al.  An Approach to Anytime Learning , 1992, ML.

[8]  Gary B. Parker,et al.  Punctuated anytime learning for hexapod gait generation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Risto Miikkulainen,et al.  Real-Time Evolution of Neural Networks in the NERO Video Game , 2006, AAAI.

[10]  Simon M. Lucas,et al.  Estimating learning rates in evolution and TDL: Results on a simple grid-world problem , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[11]  Oliver Kramer,et al.  Evolution of Human-Competitive Agents in Modern Computer Games , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[12]  Georgios N. Yannakakis,et al.  Evolving opponents for interesting interactive computer games , 2004 .