Imitating driver behavior with generative adversarial networks

The ability to accurately predict and simulate human driving behavior is critical for the development of intelligent transportation systems. Traditional modeling methods have employed simple parametric models and behavioral cloning. This paper adopts a method for overcoming the problem of cascading errors inherent in prior approaches, resulting in realistic behavior that is robust to trajectory perturbations. We extend Generative Adversarial Imitation Learning to the training of recurrent policies, and we demonstrate that our model rivals rule-based controllers and maximum likelihood models in realistic highway simulations. Our model both reproduces emergent behavior of human drivers, such as lane change rate, while maintaining realistic control over long time horizons.

[1]  Philip A. Seddon,et al.  A program for simulating the dispersion of platoons of road traffic , 1972 .

[2]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[3]  P. G. Gipps,et al.  A behavioural car-following model for computer simulation , 1981 .

[4]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[5]  Helbing,et al.  Congested traffic states in empirical observations and microscopic simulations , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[6]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[7]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[8]  Juan Zhicai,et al.  Develop a car-following model using data collected by "five-wheel system" , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[9]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[10]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[11]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[12]  Dirk Helbing,et al.  General Lane-Changing Model MOBIL for Car-Following Models , 2007 .

[13]  Robert E. Schapire,et al.  A Game-Theoretic Approach to Apprenticeship Learning , 2007, NIPS.

[14]  P. Boyraz,et al.  Signal Modelling and Hidden Markov Models for Driving Manoeuvre Recognition and Driver Fault Diagnosis in an urban road scenario , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[15]  Hussein Dia,et al.  Neural Agent Car-Following Models , 2007, IEEE Transactions on Intelligent Transportation Systems.

[16]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[17]  J. Andrew Bagnell,et al.  Efficient Reductions for Imitation Learning , 2010, AISTATS.

[18]  Jürgen Schmidhuber,et al.  Recurrent policy gradients , 2010, Log. J. IGPL.

[19]  Ali Ghaffari,et al.  A Modified Car-Following Model Based on a Neural Network Model of the Human Driver Effects , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[21]  Eduardo Mario Nebot,et al.  Estimation of Multivehicle Dynamics by Considering Contextual Information , 2012, IEEE Transactions on Robotics.

[22]  Rüdiger Dillmann,et al.  Learning context sensitive behavior models from observations for predicting traffic situations , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[23]  Christian Laugier,et al.  Comparison of parametric and non-parametric approaches for vehicle speed prediction , 2014, 2014 American Control Conference.

[24]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[25]  Qiang Liu,et al.  Vehicle lateral position prediction: A small step towards a comprehensive risk assessment system , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[26]  J A Bagnell,et al.  An Invitation to Imitation , 2015 .

[27]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[28]  David Silver,et al.  Memory-based control with recurrent neural networks , 2015, ArXiv.

[29]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Stefano Ermon,et al.  Model-Free Imitation Learning with Policy Optimization , 2016, ICML.

[31]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[32]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[33]  Stefano Ermon,et al.  Generative Adversarial Imitation Learning , 2016, NIPS.

[34]  Christian Laugier,et al.  High-speed highway scene prediction based on driver models learned from demonstrations , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[35]  Mykel J. Kochenderfer,et al.  Probabilistic Airport Acceptance Rate Prediction , 2016 .

[36]  Anca D. Dragan,et al.  Planning for Autonomous Cars that Leverage Effects on Human Actions , 2016, Robotics: Science and Systems.

[37]  Mykel J. Kochenderfer,et al.  Analysis of microscopic behavior models for probabilistic modeling of driver behavior , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[38]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[39]  Mykel J. Kochenderfer,et al.  Analysis of Recurrent Neural Networks for Probabilistic Modeling of Driver Behavior , 2017, IEEE Transactions on Intelligent Transportation Systems.