Neuroevolution of Multimodal Ms. Pac-Man Controllers Under Partially Observable Conditions

Ms. Pac-Man is a well-known video game used extensively in AI research. Past research has focused on the standard, fully observable version of Ms. Pac-Man. Recently, a partially observable variant of the game has been used in the Ms. Pac-Man Vs. Ghost Team Competition at the Computational Intelligence and Games (CIG) conference. Restricting Ms. Pac-Man’s view makes the game more challenging. Ms. Pac-Man can only see down halls within her direct line of sight. The approach to this domain presented in this paper extends an earlier approach using MM-NEAT, an algorithm for evolving modular neural networks. Experiments using several forms of evolved and human-specified modularity are presented. The best evolved agent uses a human-specified task division with output modules for different situations: no ghosts, edible ghosts, and threat ghosts. This approach placed first at the Ms. Pac-Man Vs. Ghost Team Competition at CIG 2018 against seven other competitors with an average score of 7736.63.

[1]  Rudolf Kruse,et al.  Combining cooperative and adversarial coevolution in the context of pac-man , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[2]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[3]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[4]  J. Togelius,et al.  Competitive coevolution in Ms. Pac-Man , 2013, 2013 IEEE Congress on Evolutionary Computation.

[5]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[6]  Simon M. Lucas,et al.  Using a training camp with Genetic Programming to evolve Ms Pac-Man agents , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[7]  Simon M. Lucas,et al.  Ms. Pac-Man Versus Ghost Team CIG 2016 competition , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[8]  Julian Togelius,et al.  Neuroevolution in Games: State of the Art and Open Challenges , 2014, IEEE Transactions on Computational Intelligence and AI in Games.

[9]  Romain Laroche,et al.  Hybrid Reward Architecture for Reinforcement Learning , 2017, NIPS.

[10]  Simon M. Lucas,et al.  Evolving a Neural Network Location Evaluator to Play Ms. Pac-Man , 2005, CIG.

[11]  Samad Ahmadi,et al.  Reactive control of Ms. Pac Man using information retrieval based on Genetic Programming , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[12]  Risto Miikkulainen,et al.  Discovering Multimodal Behavior in Ms. Pac-Man Through Evolution of Modular Neural Networks , 2016, IEEE Transactions on Computational Intelligence and AI in Games.

[13]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[14]  Simon M. Lucas,et al.  Pac-Man Conquers Academia: Two Decades of Research Using a Classic Arcade Game , 2018, IEEE Transactions on Games.