Deep Imitation Learning: The Impact of Depth on Policy Performance

This paper investigates the impact of network depth on the performance of imitation learning applied in the development of an end- to-end policy for controlling autonomous cars. The policy generates optimal steering commands from raw images taken from cameras attached to the car in a simulated environment. A convolutional neural network (CNN) is used to find the mapping between inputs (car images) and the desired steering angle. The CNN architecture is modified by changing the number of convolutional layers as well as the filter size. It is observed that the learned policy is capable of driving the car in the autonomous mode purely using visual information. In addition, simulation results indicate that deeper CNNs outperform shallower CNNs for learning and mimicking the human driver’s behavior. Surprisingly, the best performance is not achieved by the most complex CNN.

[1]  Abhinav Gupta,et al.  Learning to fly by crashing , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[2]  Chee Peng Lim,et al.  Medical image analysis using wavelet transform and deep belief networks , 2017, Expert Syst. Appl..

[3]  Torsten Bertram,et al.  Robot Programming by Demonstration , 2010, SIMPAR.

[4]  Mohamed Medhat Gaber,et al.  Imitation Learning , 2017, ACM Comput. Surv..

[5]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[6]  Pieter Abbeel,et al.  Autonomous Helicopter Aerobatics through Apprenticeship Learning , 2010, Int. J. Robotics Res..

[7]  Saeid Nahavandi,et al.  Trusted Autonomy Between Humans and Robots: Toward Human-on-the-Loop in Robotics and Autonomous Systems , 2017, IEEE Systems, Man, and Cybernetics Magazine.

[8]  Saeid Nahavandi,et al.  Quantifying uncertainties of neural network-based electricity price forecasts , 2013 .

[9]  Ken Goldberg,et al.  Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation , 2017, ICRA.

[10]  Saeid Nahavandi,et al.  Intent prediction of vulnerable road users from motion trajectories using stacked LSTM network , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[11]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[12]  Oliver Kramer,et al.  Evolution of Reactive Rules in Multi Player Computer Games Based on Imitation , 2005, ICNC.

[13]  Saeid Nahavandi,et al.  Lung cancer classification using deep learned features on low population dataset , 2017, 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE).

[14]  Saeid Nahavandi,et al.  Parallel deep solutions for image retrieval from imbalanced medical imaging archives , 2018, Appl. Soft Comput..

[15]  Saeid Nahavandi,et al.  Towards trusted autonomous vehicles from vulnerable road users perspective , 2017, 2017 Annual IEEE International Systems Conference (SysCon).

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[18]  Saeid Nahavandi,et al.  Medical data classification using interval type-2 fuzzy logic system and wavelets , 2015, Appl. Soft Comput..

[19]  Bernard Ghanem,et al.  Teaching UAVs to Race With Observational Imitation Learning , 2018, ArXiv.

[20]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[21]  Chrystopher L. Nehaniv,et al.  Teaching robots by moulding behavior and scaffolding the environment , 2006, HRI '06.

[22]  Bernard Ghanem,et al.  Teaching UAVs to Race Using Sim4CV , 2017 .

[23]  Sebastian Risi,et al.  DLNE: A hybridization of deep learning and neuroevolution for visual control , 2017, 2017 IEEE Conference on Computational Intelligence and Games (CIG).

[24]  Saeid Nahavandi,et al.  A neural network-GARCH-based method for construction of Prediction Intervals , 2013 .

[25]  S. Sitharama Iyengar,et al.  Data-Driven Techniques in Disaster Information Management , 2017, ACM Comput. Surv..

[26]  Daniele Loiacono,et al.  Learning drivers for TORCS through imitation using supervised methods , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[27]  Andrew Y. Ng,et al.  Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping , 1999, ICML.

[28]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[29]  Lennart Svensson,et al.  Imitation learning for vision-based lane keeping assistance , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[30]  David Silver,et al.  Applied Imitation Learning for Autonomous Navigation in Complex Natural Terrain , 2009, FSR.