Optimized sensory-motor couplings plus strategy extensions for the TORCS car racing challenge

The TORCS simulated car racing competition was introduced over a year ago. It asks for the design of racing car control strategies that have to rely on local track and driving information only, such as distance sensors, angle-to-track axis, or velocity vectors. Thus, the competition asks for strategies that are sensory-motorically grounded rather than strategies that can be designed (online or even offline) by an external observer that has full track knowledge. Moreover, the competition enforces the development of rather general driving strategies since optimization is on driving success in general rather than driving success on one particular track. This paper describes the steps taken to develop COBOSTAR, an autonomous racing car strategy with several general, context-dependent behavioral modules and strategic advancements. Most of the behavioral parameters were optimized with covariance matrix adaptation evolutionary strategy techniques. COBOSTAR won the simulated car racing competition at the IEEE Congress of Evolutionary Computation (CEC 2009) and there is still lots of room for further optimizations and strategy additions. Apart from describing the COBOSTAR racer in detail, we also outline possible next steps and future challenges.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Julian Togelius,et al.  The WCCI 2008 simulated car racing competition , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[3]  Julian Togelius,et al.  Robust player imitation using multiobjective evolution , 2009, 2009 IEEE Congress on Evolutionary Computation.

[4]  Martin V. Butz,et al.  Anticipatory Behavior in Adaptive Learning Systems, From Brains to Individual and Social Behavior [the book is a result from the third workshop on anticipatory behavior in adaptive learning systems, ABiALS 2006, Rome, Italy, September 30, 2006, colocated with SAB 2006] , 2007, ABiALS book.

[5]  V. Braitenberg.  Vehicles, Experiments in Synthetic Psychology , 1984 .

[6]  Julian Togelius,et al.  Computational Intelligence in Racing Games , 2007, Advanced Intelligent Paradigms in Computer Games.

[7]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[8]  Olivier Sigaud,et al.  Anticipatory Behavior in Adaptive Learning Systems: Foundations, Theories, and Systems , 2003 .

[9]  Julian Togelius,et al.  Evolving robust and specialized car racing skills , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[10]  Marc Ebner,et al.  Evolving driving controllers using Genetic Programming , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[11]  Risto Miikkulainen,et al.  Evolving a real-world vehicle warning system , 2006, GECCO.

[12]  Jürgen Schmidhuber,et al.  Recurrent policy gradients , 2010, Log. J. IGPL.

[13]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[14]  Christos Dimitrakakis,et al.  TORCS, The Open Racing Car Simulator , 2005 .

[15]  Martin V. Butz,et al.  Anticipatory Behavior in Adaptive Learning Systems , 2003, Lecture Notes in Computer Science.

[16]  Risto Miikkulainen,et al.  Neuroevolution of an automobile crash warning system , 2005, GECCO '05.