N 2 C : Neural Network Controller Design Using Behavioral Cloning

Modern vehicles communicate data to and from sensors, actuators, and electronic control units (ECUs) using Controller Area Network (CAN) bus, which operates on differential signaling. An autonomous ECU responsible for the execution of decision commands to an autonomous vehicle is developed by assimilating the information from the CAN bus. The conventional way of parsing the decision commands is motion planning, which uses a path tracking algorithm to evaluate the decision commands. This study focuses on designing a robust controller using behavioral cloning and motion planning of autonomous vehicle using a deep learning framework. In the first part of this study, we explore the pipeline of parsing decision commands from the path tracking algorithm to the controller and proposed a neural network-based controller (N 2 C) using behavioral cloning. The proposed network predicts throttle, brake, and torque when trained with the manual driving data acquired from the CAN bus. The efficacy of the proposed method is demonstrated by comparing the accuracy with the Proportional-Derivative-Integral (PID) controller in conjunction with the path tracking algorithm (pure pursuit and model predictive control based path follower). The second part of this study complements N 2 C, in which an end-to-end neural network for predicting the speed and steering angle is proposed with image data as an input. The performance of the proposed frameworks are evaluated in real-time and on the Udacity dataset, showing better metric scores in the former and reliable prediction in the later case when compared with the state-of-the-art methods.

[1]  Karl Zipser,et al.  MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving , 2017, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[2]  Chao Lu,et al.  Speed and steering angle prediction for intelligent vehicles based on deep belief network , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[3]  Jürgen Schmidhuber,et al.  Learning Precise Timing with LSTM Recurrent Networks , 2003, J. Mach. Learn. Res..

[4]  Moongu Jeon,et al.  Autonomous Vehicle: The Architecture Aspect of Self Driving Car , 2018, SSIP.

[5]  Andrew Simpson,et al.  Self-Driving Car Steering Angle Prediction Based on Image Recognition , 2019, ArXiv.

[6]  Peter Stone,et al.  Behavioral Cloning from Observation , 2018, IJCAI.

[7]  Yang Gao,et al.  End-to-End Learning of Driving Models from Large-Scale Video Datasets , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jiebo Luo,et al.  End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[9]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[11]  Vicente Milanés Montero,et al.  Cascade Architecture for Lateral Control in Autonomous Vehicles , 2011, IEEE Transactions on Intelligent Transportation Systems.

[12]  Sebastian Thrun,et al.  Stanley: The robot that won the DARPA Grand Challenge , 2006, J. Field Robotics.

[13]  Jiang Feng Wang,et al.  Speed Control of Tracked Vehicle Autonomous Driving System using Fuzzy Self-Tuning PID , 2019, 2019 4th International Conference on Mechanical, Control and Computer Engineering (ICMCCE).

[14]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yadong Mu,et al.  Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cues , 2017, ArXiv.

[16]  Zhiqing Huang,et al.  Learning A Steering Decision Policy for End-to-End Control of Autonomous Vehicle , 2019, 2019 5th International Conference on Control, Automation and Robotics (ICCAR).

[17]  Jiong Yang,et al.  PointPillars: Fast Encoders for Object Detection From Point Clouds , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Chunxiao Liu,et al.  Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks , 2018, AAAI.

[19]  Xiaolin Zhang,et al.  Design of Autonomous Vehicle Controller Based on BP-PID , 2019, IOP Conference Series: Earth and Environmental Science.

[20]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ali Ghaffari,et al.  A historical review on lateral and longitudinal control of autonomous vehicle motions , 2010, 2010 International Conference on Mechanical and Electrical Technology.

[22]  Ruigang Yang,et al.  The ApolloScape Dataset for Autonomous Driving , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Uwe Kiencke,et al.  Automotive Serial Controller Area Network , 1986 .

[24]  Moongu Jeon,et al.  Dynamic Control System Design for Autonomous Car , 2020, VEHITS.

[25]  J. Christian Gerdes,et al.  Autonomous Vehicle Control at the Limits of Handling , 2012 .

[26]  Maolin Chen,et al.  Behavioral cloning for driverless cars using transfer learning , 2018, 2018 IEEE/ION Position, Location and Navigation Symposium (PLANS).

[27]  Luc Van Gool,et al.  End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners , 2018, ECCV.

[28]  A. J. Healey,et al.  Adaptive sliding mode control of autonomous underwater vehicles in the dive plane , 1990 .

[29]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[30]  William Whittaker,et al.  A robust approach to high‐speed navigation for unrehearsed desert terrain , 2006, J. Field Robotics.

[31]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[32]  Chunyun Fu,et al.  Lane Detection and Trajectory Tracking Control of Autonomous Vehicle Based on Model Predictive Control , 2020 .

[33]  Vishwas Sharma,et al.  Development of model predictive motion planning and control for autonomous vehicles , 2020 .

[34]  Wei Li,et al.  Adaptive PID control of multi-DOF industrial robot based on neural network , 2020, J. Ambient Intell. Humaniz. Comput..

[35]  John E. Moody,et al.  Fast adaptive k-means clustering: some empirical results , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[36]  John F. Canny,et al.  Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Francesco Borrelli,et al.  MPC-Based Approach to Active Steering for Autonomous Vehicle Systems , 2005 .

[38]  Vicenç Puig,et al.  Autonomous vehicle control using a kinematic Lyapunov-based technique with LQR-LMI tuning , 2018 .

[39]  Eam Khwang Teoh,et al.  Fuzzy speed and steering control of an AGV , 2002, IEEE Trans. Control. Syst. Technol..

[40]  Francesco Borrelli,et al.  MPC-based yaw and lateral stabilisation via active front steering and braking , 2008 .

[41]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[42]  William Whittaker,et al.  Autonomous driving in urban environments: Boss and the Urban Challenge , 2008, J. Field Robotics.