Deep Reinforcement Learning for Autonomous Driving

Reinforcement learning has steadily improved and outperform human in lots of traditional games since the resurgence of deep neural network. However, these success is not easy to be copied to autonomous driving because the state spaces in real world are extreme complex and action spaces are continuous and fine control is required. Moreover, the autonomous driving vehicles must also keep functional safety under the complex environments. To deal with these challenges, we first adopt the deep deterministic policy gradient (DDPG) algorithm, which has the capacity to handle complex state and action spaces in continuous domain. We then choose The Open Racing Car Simulator (TORCS) as our environment to avoid physical damage. Meanwhile, we select a set of appropriate sensor information from TORCS and design our own rewarder. In order to fit DDPG algorithm to TORCS, we design our network architecture for both actor and critic inside DDPG paradigm. To demonstrate the effectiveness of our model, We evaluate on different modes in TORCS and show both quantitative and qualitative results.

[1]  Judith Hylton.  SAFE: , 1993 .

[2]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[3]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[4]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[5]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[6]  Daniel Karavolos.  Q-learning with heuristic exploration in Simulated Car Racing , 2013 .

[7]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[8]  Fernando A. Mujica,et al.  An Empirical Evaluation of Deep Learning on Highway Driving , 2015, ArXiv.

[9]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[10]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[11]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[12]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[13]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[14]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[15]  Muzamil Ahmed,et al.  Self-driving and driver relaxing vehicle , 2016, 2016 2nd International Conference on Robotics and Artificial Intelligence (ICRAI).

[16]  Jianxiong Xiao,et al.  Learning from Maps: Visual Common Sense for Autonomous Driving , 2016, ArXiv.

[17]  Koray Kavukcuoglu,et al.  PGQ: Combining policy gradient and Q-learning , 2016, ArXiv.

[18]  D. Cremers,et al.  Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks , 2016, ArXiv.

[19]  Chung Choo Chung,et al.  Autonomous braking system via deep reinforcement learning , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[20]  Wenshuo Wang,et al.  Feature analysis and selection for training an end-to-end autonomous vehicle controller using deep learning approach , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[21]  Cewu Lu,et al.  Virtual to Real Reinforcement Learning for Autonomous Driving , 2017, BMVC.

[22]  David Isele,et al.  Navigating Intersections with Autonomous Vehicles using Deep Reinforcement Learning , 2017 .

[23]  David Isele,et al.  Navigating Occluded Intersections with Autonomous Vehicles Using Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).