Autonomous vehicle navigation using evolutionary reinforcement learning

Reinforcement learning schemes perform direct on-line search in control space. This makes them appropriate for modifying control rules to obtain improvements in the performance of a system. The effectiveness of a reinforcement learning strategy is studied here through the training of a learning classifier system (LCS) that controls the movement of an autonomous vehicle in simulated paths including left and right turns. The LCS comprises a set of condition-action rules (classifiers) that compete to control the system and evolve by means of a genetic algorithm (GA). Evolution and operation of classifiers depend upon an appropriate credit assignment mechanism based on reinforcement learning. Different design options and the role of various parameters have been investigated experimentally. The performance of vehicle movement under the proposed evolutionary approach is superior compared with that of other (neural) approaches based on reinforcement learning that have been applied previously to the same benchmark problem.

[1]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[2]  Andreas Stafylopatis,et al.  Collision-Free Movement of an Autonomous Vehicle Using Reinforcement Learning , 1992, ECAI.

[3]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[4]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[5]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[6]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[7]  Paul Thagard,et al.  Induction: Processes Of Inference , 1989 .

[8]  D. E. Goldberg,et al.  Optimization and Machine Learning , 2022 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning is Direct Adaptive Optimal Control , 1992, 1991 American Control Conference.

[10]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[11]  Larry Bull,et al.  A zeroth level corporate classifier system , 1999 .

[12]  Mohamad H. Hassoun,et al.  Neurocontrollers trained with rules extracted by a genetic assisted reinforcement learning system , 1995, IEEE Trans. Neural Networks.

[13]  Stewart W. Wilson ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[14]  Hideyuki Takagi,et al.  Genetic Algorithms and Fuzzy Logic , 1998 .

[15]  J. Millán,et al.  A Reinforcement Connectionist Approach to Robot Path Finding in Non-Maze-Like Environments , 2004, Machine Learning.

[16]  David E. Goldberg,et al.  Implicit Niching in a Learning Classifier System: Nature's Way , 1994, Evolutionary Computation.

[17]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[18]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[19]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[20]  Marco Dorigo,et al.  A comparison of Q-learning and classifier systems , 1994 .