End-to-end learning for high-precision lane keeping via multi-state model

High-precision lane keeping is essential for the future autonomous driving. However, due to the imbalanced and inaccurate datasets collected by human drivers, current end-to-end driving models have poor lane keeping the effect. To improve the precision of lane keeping, this study presents a novel multi-state model-based end-to-end lane keeping method. First, three driving states will be defined: going straight, turning right and turning left. Second, the finite-state machine (FSM) table as well as three kinds of training datasets will be generated based on the three driving states. Instead of collecting the dataset by human drivers, the accurate dataset will be collected by the high-performance path following controller. Third, three sets of parameters based on 3DCNN-LSTM model will be trained for going straight, turning left and turning right, which will be combined with FSM table to form a multi-state model. This study evaluates the multi-state model by testing it on five tracks and recording the lane keeping error. The result shows the multi-state model-based end-to-end method performs the higher precision of lane keeping than the traditional single end-to-end model.

[1]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[4]  Jarrod M. Snider.  Automatic Steering Methods for Autonomous Automobile Path Tracking , 2009 .

[5]  Vasu Alagar,et al.  Extended Finite State Machine , 2011 .

[6]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[11]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[12]  Emilio Frazzoli,et al.  A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles , 2016, IEEE Transactions on Intelligent Vehicles.

[13]  Xinyu Zhang,et al.  A study on key technologies of unmanned driving , 2016, CAAI Trans. Intell. Technol..

[14]  Sascha Lange,et al.  Predicting Time Series with Space-Time Convolutional and Recurrent Neural Networks , 2017, ESANN.

[15]  Heng Tao Shen,et al.  Beyond Frame-level CNN: Saliency-Aware 3-D CNN With LSTM for Video Action Recognition , 2017, IEEE Signal Processing Letters.

[16]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Wenshuo Wang,et al.  Feature analysis and selection for training an end-to-end autonomous vehicle controller using deep learning approach , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[18]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Qiang Wu,et al.  A 3D-CNN based video hashing method , 2018, International Conference on Digital Image Processing.

[20]  Narciso García,et al.  Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Alexander J. Smola,et al.  Compressed Video Action Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Cewu Lu,et al.  LiDAR-Video Driving Dataset: Learning Driving Policies Effectively , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Jing Li,et al.  Two-class 3D-CNN classifiers combination for video copy detection , 2018, Multimedia Tools and Applications.