Acquiring Visibly Intelligent Behavior with Example-Guided Neuroevolution

Much of artificial intelligence research is focused on devising optimal solutions for challenging and well-defined but highly constrained problems. However, as we begin creating autonomous agents to operate in the rich environments of modern videogames and computer simulations, it becomes important to devise agent behaviors that display the visible attributes of intelligence, rather than simply performing optimally. Such visibly intelligent behavior is difficult to specify with rules or characterize in terms of quantifiable objective functions, but it is possible to utilize human intuitions to directly guide a learning system toward the desired sorts of behavior. Policy induction from human-generated examples is a promising approach to training such agents. In this paper, such a method is developed and tested using Lamarckian neuroevolution. Artificial neural networks are evolved to control autonomous agents in a strategy game. The evolution is guided by human-generated examples of play, and the system effectively learns the policies that were used by the player to generate the examples. I.e., the agents learn visibly intelligent behavior. In the future, such methods are likely to play a central rule in creating autonomous agents for complex environments, making it possible to generate rich behaviors derived from nothing more formal than the intuitively generated example, of designers, players, or subject-matter experts.

[1]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[2]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[3]  Risto Miikkulainen,et al.  Neuroevolution for adaptive teams , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[4]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[7]  Claude Sammut,et al.  Learning to Fly , 1992, ML.

[8]  Maja J. Matarić,et al.  A framework for learning from demonstration, generalization and practice in human-robot domains , 2003 .

[9]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[10]  Lakhmi C. Jain,et al.  Computational Intelligence in Games , 2005, IEEE Transactions on Neural Networks.

[11]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[12]  John E. Laird,et al.  Learning procedural knowledge through observation , 2001, K-CAP '01.

[13]  W. M. Jenkins,et al.  Genetic Algorithms and Neural Networks , 1999, Neural Networks in the Analysis and Design of Structures.

[14]  Risto Miikkulainen,et al.  Exploiting Sensor Symmetries in Example-based Training for Intelligent Agents , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[15]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[17]  Larry R. Medsker,et al.  Genetic Algorithms and Neural Networks , 1995 .

[18]  Claude Sammut,et al.  A Framework for Behavioural Cloning , 1995, Machine Intelligence 15.

[19]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[20]  Jacques Periaux,et al.  Genetic Algorithms in Engineering and Computer Science , 1996 .

[21]  Risto Miikkulainen,et al.  Evolving visibly intelligent behavior for embedded game agents , 2006 .

[22]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[23]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[24]  Wan-Chi Siu,et al.  A study of the Lamarckian evolution of recurrent neural networks , 2000, IEEE Trans. Evol. Comput..

[25]  Simon M. Lucas,et al.  Cellz: a simple dynamic game for testing evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[26]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[27]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[28]  Risto Miikkulainen,et al.  Computational Intelligence in Games , 2006 .