UT2: Human-like behavior via neuroevolution of combat behavior and replay of human traces

The UT⁁2 bot, which had a humanness rating of 27.2727% in BotPrize 2010, is based on two core ideas: (1) multiobjective neuroevolution is used to learn skilled combat behavior, but filters on the available combat actions ensure that the behavior is still human-like despite being evolved for performance, and (2) a database of traces of human play is used to help the bot get unstuck when its navigation capabilities fail. Several changes have recently been made to UT⁁2: Extra input features have been provided to the bot to help it evolve better combat behavior, the role of human traces in the navigation of the bot has been expanded, and an extra control module has been added which encourages the bot to observe other players the way a human would, rather than simply battle them. These changes should make UT⁁2 act more human-like in this year's BotPrize competition.

[1]  H. Pao,et al.  Game Bot Detection via Avatar Trajectory Analysis , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[2]  Yago Sáez,et al.  Chuck Norris rocks! , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[3]  Vadim Bulitko,et al.  An evaluation of models for predicting opponent positions in first-person shooter video games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[4]  Philip Hingston,et al.  A new design for a Turing Test for Bots , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[5]  Philip Hingston,et al.  A Turing Test for Computer Game Bots , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[6]  Risto Miikkulainen,et al.  Automatic feature selection in neuroevolution , 2005, GECCO '05.

[7]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[8]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[9]  Risto Miikkulainen,et al.  Believable Bot Navigation via Playback of Human Traces , 2012, Believable Bots.

[10]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[11]  Frank Dignum,et al.  Evolutionary neural networks for Non-Player Characters in Quake III , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[12]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[13]  Julian Togelius,et al.  Hierarchical controller learning in a First-Person Shooter , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[14]  Ross Graham,et al.  Realistic Agent Movement in Dynamic Game Environments , 2005, DiGRA Conference.

[15]  G. L. Dirichlet.  Über die Reduction der positiven quadratischen Formen mit drei unbestimmten ganzen Zahlen. , 1850 .

[16]  Abdennour El Rhalibi,et al.  Machine learning techniques for FPS in Q3 , 2004, ACE '04.

[17]  Christian Bauckhage,et al.  Synthesizing Movements for Computer Game Characters , 2004, DAGM-Symposium.

[18]  Risto Miikkulainen,et al.  Human-Like Combat Behaviour via Multiobjective Neuroevolution , 2012, Believable Bots.

[19]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[20]  R. Thawonmas,et al.  Outline of ICE-CEC2011 and Its Mechanism for Learning FPS Tactics , 2011 .

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  Risto Miikkulainen,et al.  Integration and Evaluation of Exploration-Based Learning in Games , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[23]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[24]  Joanna J. Bryson,et al.  Intelligence by design: principles of modularity and coordination for engineering complex adaptive agents , 2001 .

[25]  Daniele Loiacono,et al.  Learning drivers for TORCS through imitation using supervised methods , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[26]  Georges Voronoi.  Nouvelles applications des paramètres continus à la théorie des formes quadratiques. Premier mémoire. Sur quelques propriétés des formes quadratiques positives parfaites. , 1908 .

[27]  Risto Miikkulainen,et al.  Evolving agent behavior in multiobjective domains using fitness-based shaping , 2010, GECCO '10.

[28]  Christopher G. Atkeson,et al.  Transfer of policies based on trajectory libraries , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Christopher G. Atkeson,et al.  Policies based on trajectory libraries , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[30]  José María Valls,et al.  Programming Robosoccer agents by modeling human behavior , 2009, Expert Syst. Appl..

[31]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.