Imitation Learning for End to End Vehicle Longitudinal Control with Forward Camera

In this paper we present a complete study of an end-to-end imitation learning system for speed control of a real car, based on a neural network with a Long Short Term Memory (LSTM). To achieve robustness and generalization from expert demonstrations, we propose data augmentation and label augmentation that are relevant for imitation learning in longitudinal control context. Based on front camera image only, our system is able to correctly control the speed of a car in simulation environment, and in a real car on a challenging test track. The system also shows promising results in open road context.

[1]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[2]  Jiebo Luo,et al.  End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[3]  Sebastian Klemm,et al.  Integrating end-to-end learned steering into probabilistic autonomous driving , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[4]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Javier Alonso-Mora,et al.  Planning and Decision-Making for Autonomous Vehicles , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[6]  Mariusz Bojarski,et al.  VisualBackProp: visualizing CNNs for autonomous driving , 2016, ArXiv.

[7]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[8]  Sebastian Thrun,et al.  Path Planning for Autonomous Vehicles in Unknown Semi-structured Environments , 2010, Int. J. Robotics Res..

[9]  Johann Marius Zöllner,et al.  Adding navigation to the equation: Turning decisions for end-to-end vehicle control , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[10]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[11]  Shu Yang,et al.  Baidu driving dataset and end-to-end reactive control model , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[12]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.

[13]  Fabien Moutarde,et al.  End to End Vehicle Lateral Control Using a Single Fisheye Camera , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).