Evolving large-scale neural networks for vision-based TORCS

The TORCS racing simulator has become a standard testbed used in many recent reinforcement learning competitions, where an agent must learn to drive a car around a track using a small set of task-specific features. In this paper, large, recurrent neural networks (with over 1 million weights) are evolved to solve a much more challenging version of the task that instead uses only a stream of images from the driver’s perspective as input. Evolving such large nets is made possible by representing them in the frequency domain as a set of coefficients that are transformed into weight matrices via an inverse Fourier-type transform. To our knowledge this is the first attempt to tackle TORCS using vision, and successfully evolve a neural network controllers of this size.

[1]  Jürgen Schmidhuber,et al.  Evolving neural networks in compressed weight space , 2010, GECCO '10.

[2]  W. Marsden.  I and J , 2012 .

[3]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[4]  Risto Miikkulainen,et al.  Accelerated Neural Evolution through Cooperatively Coevolved Synapses , 2008, J. Mach. Learn. Res..

[5]  Lawrence G. Roberts,et al.  Machine Perception of Three-Dimensional Solids , 1963, Outstanding Dissertations in the Computer Sciences.

[6]  Jürgen Schmidhuber,et al.  A Frequency-Domain Encoding for Neuroevolution , 2012, ArXiv.

[7]  Kenneth O. Stanley,et al.  A novel generative encoding for exploiting neural network sensor and output geometry , 2007, GECCO '07.

[8]  Julian Togelius,et al.  The 2009 Simulated Car Racing Championship , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[9]  Julian Togelius,et al.  The WCCI 2008 simulated car racing competition , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[10]  Jürgen Schmidhuber,et al.  Compressed Network Complexity Search , 2012, PPSN.

[11]  Jürgen Schmidhuber,et al.  Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability , 1997, Neural Networks.

[12]  Y. Engel,et al.  A dynamical model of the octopus arm , 1997, Neuroscience Letters.

[13]  Jürgen Schmidhuber,et al.  Searching for Minimal Neural Networks in Fourier Space , 2010, AGI 2010.

[14]  Daniele Loiacono,et al.  Simulated Car Racing Championship: Competition Software Manual , 2013, ArXiv.

[15]  Tamar Flash,et al.  Dynamic model of the octopus arm. I. Biomechanics of the octopus reaching movement. , 2005, Journal of neurophysiology.

[16]  Jürgen Schmidhuber,et al.  Generalized compressed network search , 2012, GECCO '12.