A platform for evolving intelligently interactive adversaries.

Entertainment software developers face significant challenges in designing games with broad appeal. One of the challenges concerns creating nonplayer (computer-controlled) characters that can adapt their behavior in light of the current and prospective situation, possibly emulating human behaviors. This adaptation should be inherently novel, unrepeatable, yet within the bounds of realism. Evolutionary algorithms provide a suitable method for generating such behaviors. This paper provides background on the entertainment software industry, and details a prior and current effort to create a platform for evolving nonplayer characters with genetic and behavioral traits within a World War I combat flight simulator.

[1]  F. Corno,et al.  Exploiting co-evolution and a modified island model to climb the Core War hill , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[2]  Jordan B. Pollack,et al.  A game-theoretic and dynamical-systems analysis of selection methods in coevolution , 2005, IEEE Transactions on Evolutionary Computation.

[3]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[4]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[5]  David B. Fogel,et al.  On the instability of evolutionary stable strategies in small populations , 1998 .

[6]  Jim Esch,et al.  A Self-Learning Evolutionary Chess Program , 2004, Proc. IEEE.

[7]  Marcus Gallagher,et al.  Learning to play Pac-Man: an evolutionary, rule-based approach , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[8]  Xin Yao,et al.  The impact of noise on iterated prisoner's dilemma with multiple levels of cooperation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[9]  Paul J. Darwen,et al.  Why co-evolution beats temporal difference learning at Backgammon for a linear architecture, but not a non-linear architecture , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[10]  D. Fogel,et al.  On the instability of evolutionary stable strategies. , 1997, Bio Systems.

[11]  Alex Lubberts and Risto Miikkulainen.  Co-Evolving a Go-Playing Neural network , 2001 .

[12]  David B. Fogel,et al.  Evolving neural networks to play checkers without relying on expert knowledge , 1999, IEEE Trans. Neural Networks.

[13]  David B. Fogel.  Evolving strategies in blackjack , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[14]  Kyung-Joong Kim,et al.  Systematically incorporating domain-specific knowledge into evolutionary speciated checkers players , 2005, IEEE Transactions on Evolutionary Computation.

[15]  D. Fogel,et al.  Evolving continuous behaviors in the Iterated Prisoner's Dilemma. , 1996, Bio Systems.

[16]  Ivan Tanev,et al.  Evolution of the driving styles of anticipatory agent remotely operating a scaled model of racing car , 2005, 2005 IEEE Congress on Evolutionary Computation.

[17]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[18]  Steve Grand,et al.  Creation: Life and How to Make It , 2001 .

[19]  Graham Kendall,et al.  An evolutionary approach for the tuning of a chess evaluation function using population dynamics , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[20]  Tughrul Arslan,et al.  Proceedings Of The 2000 Congress On Evolutionary Computation , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[21]  E. Hughes.  Piece difference: simple to evolve? , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[22]  David B. Fogel,et al.  Evolving Behaviors in the Iterated Prisoner's Dilemma , 1993, Evolutionary Computation.

[23]  David B. Fogel,et al.  A platform for evolving characters in competitive games , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[24]  J. Reed,et al.  Simulation of biological evolution and machine learning. I. Selection of self-reproducing numeric patterns by data processing machines, effects of hereditary control, mutation type and crossing. , 1967, Journal of theoretical biology.

[25]  Xin Yao,et al.  Behavioral diversity, choices and noise in the iterated prisoner's dilemma , 2005, IEEE Transactions on Evolutionary Computation.

[26]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[27]  Sung-Bae Cho,et al.  Evolving Reactive NPCs for the Real-Time Simulation Game , 2005, CIG.

[28]  Simon M. Lucas,et al.  Coevolution versus self-play temporal difference learning for acquiring position evaluation in small-board go , 2005, IEEE Transactions on Evolutionary Computation.

[29]  Giovanni Squillero,et al.  Evolving assembly programs: how games help microprocessor validation , 2005, IEEE Transactions on Evolutionary Computation.

[30]  D. Fogel,et al.  Multiple-vector self-adaptation in evolutionary algorithms. , 2001, Bio Systems.

[31]  David B. Fogel,et al.  On the Relationship between the Duration of an Encounter and the Evolution of Cooperation in the Iterated Prisoner's Dilemma , 1995, Evolutionary Computation.

[32]  Evan J. Hughes.  Checkers using a co-evolutionary on-line evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[33]  Gary B. Parker,et al.  Evolution and prioritization of survival strategies for a simulated robot in Xpilot , 2005, 2005 IEEE Congress on Evolutionary Computation.

[34]  Xin Yao,et al.  Does extra genetic diversity maintain escalation in a co-evolutionary arms race , 2000 .

[35]  Clark C. Guest,et al.  Planning an endgame move set for the game RISK: a comparison of search algorithms , 2005, IEEE Transactions on Evolutionary Computation.

[36]  David B. Fogel,et al.  Evolution, neural networks, games, and intelligence , 1999, Proc. IEEE.

[37]  Sean Luke,et al.  Genetic Programming Produced Competitive Soccer Softbot Teams for RoboCup97 , 1998 .

[38]  Proceedings of the 2005 IEEE Symposium on Computational Intelligence and Games (CIG05), Essex University, Colchester, Essex, UK, 4-6 April, 2005 , 2005, CIG.

[39]  Giovanni Squillero,et al.  On the evolution of corewar warriors , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[40]  D.B. Fogel,et al.  A self-learning evolutionary chess program , 2004, Proceedings of the IEEE.

[41]  Sushil J. Louis,et al.  Playing to learn: case-injected genetic algorithms for learning to play computer games , 2005, IEEE Transactions on Evolutionary Computation.

[42]  Thomas Bäck,et al.  Evolutionary computation: comments on the history and current state , 1997, IEEE Trans. Evol. Comput..

[43]  Graham Kendall,et al.  An investigation of an evolutionary approach to the opening of Go , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[44]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[45]  Sung-Bae Cho,et al.  Evolving speciated checkers players with crowding algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[46]  Hisao Ishibuchi,et al.  Evolution of iterated prisoner's dilemma game strategies in structured demes under random pairing in game playing , 2005, IEEE Transactions on Evolutionary Computation.

[47]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[48]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization approaches to coevolve strategies for the iterated prisoner's dilemma , 2005, IEEE Transactions on Evolutionary Computation.

[49]  D B Fogel,et al.  Do evolutionary processes minimize expected losses? , 2000, Journal of theoretical biology.

[50]  Jordan B. Pollack,et al.  Co-Evolution in the Successful Learning of Backgammon Strategy , 1998, Machine Learning.

[51]  Risto Miikkulainen,et al.  Discovering Complex Othello Strategies through Evolutionary Neural Networks , 1995, Connect. Sci..

[52]  Simon M. Lucas,et al.  Evolving a Neural Network Location Evaluator to Play Ms. Pac-Man , 2005, CIG.

[53]  Colin Frayn.  An Evolutionary Approach to Strategies for the Game of Monopoly , 2005, CIG.

[54]  Graham Kendall,et al.  The evolution of blackjack strategies , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[55]  Risto Miikkulainen,et al.  Evolving Neural Networks to Play Go , 2004, Applied Intelligence.

[56]  Julian Togelius,et al.  Evolving controllers for simulated car racing , 2005, 2005 IEEE Congress on Evolutionary Computation.