Backpropagation without human supervision for visual control in Quake II

Backpropagation and neuroevolution are used in a Lamarckian evolution process to train a neural network visual controller for agents in the Quake II environment. In previous work, we hand-coded a non-visual controller for supervising in backpropagation, but hand-coding can only be done for problems with known solutions. In this research the problem for the agent is to attack a moving enemy in a visually complex room with a large central pillar. Because we did not know a solution to the problem, we could not hand-code a supervising controller; instead, we evolve a non-visual neural network as supervisor to the visual controller. This setup creates controllers that learn much faster and have a greater fitness than those learning by neuroevolution-only on the same problem in the same amount of time.

[1]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[2]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[3]  Daniel Thalmann,et al.  A vision-based approach to behavioural animation , 1990, Comput. Animat. Virtual Worlds.

[4]  Dean Pomerleau,et al.  Efficient Training of Artificial Neural Networks for Autonomous Navigation , 1991, Neural Computation.

[5]  John J. Grefenstette,et al.  Lamarckian Learning in Multi-Agent Environments , 1991, ICGA.

[6]  Shumeet Baluja,et al.  Evolution of an artificial neural network based autonomous land vehicle controller , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Wan-Chi Siu,et al.  A study of the Lamarckian evolution of recurrent neural networks , 2000, IEEE Trans. Evol. Comput..

[8]  Steve C. Maddock,et al.  Using Synthetic Vision for Autonomous Non-Player Characters , 2003, Inteligencia Artif..

[9]  A. Watt,et al.  ARTÍCULO Using Synthetic Vision for Autonomous Non-Player Characters in Computer Games , 2003 .

[10]  Christian Bauckhage,et al.  Learning Human-Like Opponent Behavior for Interactive Computer Games , 2003, DAGM-Symposium.

[11]  L. D. Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[12]  Dario Floreano,et al.  Coevolution of active vision and feature selection , 2004, Biological Cybernetics.

[13]  Abdennour El Rhalibi,et al.  Machine learning techniques for FPS in Q3 , 2004, ACE '04.

[14]  Christian Bauckhage,et al.  Learning Human-Like Movement Behavior for Computer Games , 2004 .

[15]  Ross Graham,et al.  Neural Pathways for Real-Time Dynamic Computer Games , 2005 .

[16]  Risto Miikkulainen,et al.  Evolving a real-world vehicle warning system , 2006, GECCO.

[17]  Gary B. Parker,et al.  Using a Queue Genetic Algorithm to Evolve Xpilot Control Strategies on a Distributed System , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[18]  Risto Miikkulainen,et al.  Acquiring Visibly Intelligent Behavior with Example-Guided Neuroevolution , 2007, AAAI.

[19]  Bobby D. Bryant,et al.  Neuro-visual control in the Quake II game engine , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[20]  Bobby D. Bryant,et al.  Visual control in quake II with a cyclic controller , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[21]  Bobby D. Bryant,et al.  Lamarckian neuroevolution for visual control in the Quake II environment , 2009, 2009 IEEE Congress on Evolutionary Computation.